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Task  Difficulty  Measurement  by  the 
United  States  Air  Force  Occupational  Measurement  Program1 


INTRODUCTION 

One  effective  method  of  enhancing  the  effectiveness  and  dealing  with  problems 
relating  to  human  resource  development  and  management  in  organizations  is,  from  a 
psychological  point  of  view,  to  understand  the  behaviors  of  individuals  in  the 
organizations.  To  understand  individual  behaviors,  it  is  necessary  to  first  know  and 
understand  their  jobs.  Job  analysis  is  a  means  for  providing  this  understanding. 

Job  analysis  research  and  application  are  relatively  recent  phenomenon  in  the 
world  of  industrial  psychology.  Extensive  research  primarily  performed  by  the  United 
States  military  has  resulted  in  successful  job  analysis  programs  in  both  the  military  and 
civilian  sectors.  The  job  analysis  program  operated  by  the  United  States  Air  Force 
Occupational  Measurement  Squadron  (USAFOMS)  is  responsible  for  providing  objective 
data  to  aid  Air  Force  managers  in  making  important  training  and  personnel  management 
decisions.  To  accomplish  this  mission,  a  wide  array  of  occupational  data  are  collected 
and  analyzed  using  the  Comprehensive  Occupational  Data  Analysis  Program  (CODAP) 
(Christal,  1974).  One  specific  task  factor,  referred  to  as  "task  difficulty,"  has  wide 
acceptance  for  such  diverse  purposes  as  establishing  task  training  priorities  and  setting 
aptitude  requirements  for  jobs.  However,  because  of  the  current  procedures  used  by 
USAFOMS  in  collecting  task  difficulty  data,  the  reliability  and  validity  of  the  data  being 
collected  are  in  question,  making  the  topic  of  USAFOMS's  task  difficulty  data  a  highly 
controversial  issue  in  the  Air  Force  community. 

Task  difficulty  (TD)  is  a  generic  term  used  to  describe  measures  of  task 
performance  difficulty  and  task  learning  difficulty.  Research  indicates  that  defining  task 
difficulty  in  terms  of  task  learning  difficulty  as  opposed  to  task  performance  difficulty 
improves  the  rating  reliability  of  survey  respondents  (Cragun  &  McCormick,  1967; 

Lecznar,  1971;  Mead  1970a;  Mead  1970b;  Mead  &  Christal,  1970;  Weeks,  1984).  The 
current  definition  of  task  difficulty  as  used  by  USAFOMS  in  the  United  States  Air  Force 
(USAF)  Occupational  Surveys  is  "the  amount  of  time  needed  to  learn  to  do  a  task 
satisfactorily."  However,  the  lack  of  clear  distinct  instructions  highlighting  this  definition  in 
the  data  collection  instrument  may  be  impacting  the  validity  of  the  data  being  collected. 
Respondents  may  not  be  providing  learning  difficulty  but  perhaps  performance  difficulty  or 
a  variation  of  both.  The  current  research  goals  are  to  identify  the  magnitude  and  type  of 
error  associated  with  the  current  method  of  task  difficulty  data  collection  and  to  develop  a 
technique  to  minimize  errors  resulting  from  the  current  task  difficulty  instructions. 


1  Revised  version  of  senior  author’s  thesis  (Boyce,  1994)  written  while  a  member  of  the 
USAF  Occupational  Measurement  Squadron,  Randolph  AFB,  Tx.  The  second  author 
was  the  supervising  professor. 


A  discussion  of  task  difficulty,  as  collected  and  used  by  the  USAF  job  analysis 
program,  is  presented  in  the  following  sections.  The  discussion  begins  with  an  overview 
of  job  analysis,  is  followed  by  descriptions  of  CODAP  and  the  Air  Force  job  analysis 
program,  and  concludes  with  discussions  of  the  uses,  definitions,  and  data  collection  and 
analysis  techniques  for  the  task  difficulty  measures.  The  sections  are  organized  to  flow 
from  a  brief  account  of  job  analysis  to  the  specific  research  necessary  to  validate 
USAFOMS’  definition  of  task  difficulty  and  to  assess  the  reliability  of  information  being 
collected. 


Job  Analysis  Overview 

Job  Analysis  serves  as  the  framework  for  organizational  decisions,  work  and 
equipment  design,  and  human-resources  management.  As  defined  by  Jewell  (1985),  job 
analysis  is  a  "procedure  by  which  information  about  job  tasks  and  requirement  is  obtained 
through  formal  methods  of  data  collection  and  analysis  (p.  247)."  The  results  of  this 
procedure  provide  information  about  the  job  itself  (work  tasks,  equipment  used),  the 
outcome  of  worker  activities  (products  made,  services  performed),  working  conditions 
(organizational  structure,  work  schedule),  and  human  requirements  (education/training, 
physical  requirements).  The  end  product  of  a  job  analysis  is  used  to  help  organizations 
carry  out  a  variety  of  individual,  organizational,  and  research  oriented  activities,  such  as 
employee  development,  test  validation,  job  classification,  personnel  recruiting,  selection, 
placement,  and  training  (Cascio,  1991;  Jewell,  1985;  McCormick  &  I Igen,  1985). 

Of  the  several  methods  available  to  analyze  a  job,  structured  job  analysis 
questionnaires  have  evolved  as  the  most  systematic  technique  of  collecting  quantitative 
information.  Position  Analysis  Questionnaire  (PAQ),  Functional  Job  Analysis  (FJA),  and 
task  inventories  are  three  of  the  more  popular  structured  questionnaires.  A  study 
conducted  by  Levine,  Ash,  Hall,  and  Sistrunk  (1983)  evaluated  the  effectiveness  and 
practicality  of  seven  job  analysis  methods.  Though  the  study  found  that  job  analysis 
methods  are  perceived  as  differentially  effective  for  various  purposes,  in  general,  the 
PAQ,  FJA,  and  task  inventory-CODAP  received  consistently  high  ratings.  The  PAQ  and 
FJA  consists  of  worker-oriented  work  activity  items,  while  the  task  inventory  consists  of 
job-oriented  work  activity  items  and  hence  is  also  referred  to  as  a  job  inventory. 

The  PAQ,  developed  by  McCormick,  Jeanneret,  and  Mecham  at  Purdue 
University,  consists  of  194  job  elements  in  six  divisions.  The  six  divisions  are  information 
input,  mental  processes,  worker  output,  relationship  with  others,  job  context,  and  other 
job  characteristics.  Individual  job  elements  within  each  of  the  six  groups  are  rated  on 
appropriate  scales,  such  as  possibility  of  occurrence,  importance  to  the  job,  amount  of 
time,  extent  of  use,  and  applicability  (Jewell,  1985;  McCormick,  Jeanneret,  &  Mecham, 
1972).  Results  of  the  questionnaire  can  be  used  to  develop  an  overall  profile  of  a  job  in 
terms  of  12  dimensions.  The  12  basic  job  dimensions  are  indicated  in  Figure  1.  Rather 
than  describing  a  job  in  terms  of  the  194  elements,  a  job  can  be  described  as  high  or  low 
on  such  dimensions  as  "3,"  performing  clerical  related  activities,  "7,"  performing  routine 
repetitive  activities,  and  "9," 
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The  12  Basic  Job  Dimensions  of  the  Position  Analysis  Questionnaire 

1 .  Having  decision  making/communicating/general  responsibilities 

2.  Operating  machines  and  equipment 

3.  Performing  clerical  related  activities 

4.  Performing  technical  related  activities 

5.  Performing  service  related  activities 

6.  Working  regular  day  versus  other  work  schedules 

7.  Performing  routine  repetitive  activities 

8.  Being  aware  of  work  environment 

9.  Engaging  in  physical  activity 

10.  Supervising/Coordinating  other  personnel 

11.  Public  and  customer  related  contacts 

12.  Working  in  an  unpleasant,  hazardous,  demanding  environment _ 

Figure  1.  PAQ's  12  Job  Dimensions 

engaging  in  physical  activities.  The  final  results  can  then  be  compared  across  different 
jobs.  Despite  the  PAQ's  weaknesses,  such  as  having  a  relatively  high  reading  level  with 
a  blue  collar  orientation  and  lack  of  specific  work  activity  descriptions,  it  has  been  used 
successfully  in  several  work  settings  in  many  countries  (McCormick  &  llgen,  1985). 

The  FJA  procedure,  developed  by  the  United  States  Training  and  Employment 
Service  (USfES),  attempts  to  identify  exactly  what  the  worker  does  in  the  job  as  well  as 
the  results  of  the  worker's  behavior  (Cascio,  1991;  Olson,  Fine,  Myers,  &  Jennings,  1981; 
Schmitt  &  Fine,  1983).  Each  job  is  assigned  a  code  that  depicts  a  broad  action  that 
summarizes  what  the  worker  does  in  relation  to  data,  people,  or  things.  As  shown  in 
Figure  2,  tasks  are  described  on  a  behavior  observation  worksheet  which  consists  of  four 
general  sections:  what,  why,  how,  and  worker  functions.  "What"  describes  what  the 
worker  does  and  to  whom  or  to  what.  "Why"  describes  the  purpose  of  the  worker's  action 
and  "how"  describes  the  tools,  equipment,  or  work  aids  used  to  accomplish  the  task  as 
well  as  the  nature  and  source  of  instructions.  "Worker  functions"  describe  the  orientation 
and  level  of  worker  activity  with  data,  people,  and  things.  Percentages  are  used  to 
indicate  the  relative  amount  of  involvement  with  these  functions.  These  level  and 
orientation  measures  can  be  applied  to  all  tasks,  and  respectively  to  all  jobs.  Therefore, 
the  worker  function  scales  provide  a  way  of  comparing  all  tasks  and  jobs  on  a  common 
basis.  The  results  are  published  by  the  United  States  Department  of  Labor  in  the 
Dictionary  of  Occupational  Titles  (DOT)  which  serves  as  a  major  job  information 
reference  source. 

During  the  mid-1930’s,  DOT  was  developed  by  the  US  Employment  Service 
(USES)  in  response  to  the  demand  of  an  expanding  public  employment  service  for 
standardized  occupational  information  to  support  job  placement  activities.  The  first 
edition,  published  in  1939,  contained  approximately  17,500  occupations  placed  into  one 
of  550  occupational  groups.  Four  editions  later,  the  1991  revision  includes  standardized, 
comprehensive  descriptions  of  job  duties  and  related  information  for  979  occupational 
groups  covering  over  20,000  military  and  civilian  occupations  or  nearly  all  jobs  in  the  U.S. 
economy  (Dictionary,  1991). 
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Figure  2.  FJA  Behavior  Observation  Worksheet 


The  DOT  occupational  classification  system  organizes  groups  of  jobs  into 
occupations  based  on  the  interrelationships  of  job  tasks  and  requirements.  Each 
occupation  is  defined  in  a  systematic  fashion.  There  are  seven  parts  to  each 
occupational  definition:  1)  Occupational  Code  Number,  2)  Occupational  Title,  3)  Industry 
Designation,  4)  Alternate  Titles,  5)  Body  of  the  Definition  which  includes  a  lead  statement, 
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task's  element  statements,  and  "may"  items;  6)  Undefined  Related  Titles,  and  7) 

Definition  Trailer.  The  occupational  code  number  consists  of  nine  digits.  The  first  three 
digits  identify  a  particular  occupational  group.  The  middle  three  digits  are  the  FJA  Worker 
Functions  ratings  of  the  tasks  performed  in  the  occupation.  The  last  three  digits  serve  to 
differentiate  occupations.  These  full  nine  digits  provide  each  occupation  with  a  unique 
code  which  when  used  in  conjunction  with  the  Handbook  of  Occupational  Keywords  and 
Occupational  Outlook  Handbook,  both  issued  by  Department  of  Labor,  are  extremely 
useful  for  providing  a  familiarization  with  the  vast  array  of  jobs  and  terminology 
associated  with  each  job,  for  identifying  activities  and  trade  requirements  of  jobs,  and  for 
identifying  transfer  and  promotional  patterns  within  particular  occupational  specialties 
(Cascio,  1991;  Dictionary,  1991). 

Task  or  job  inventories  typically  consist  of  lists  of  tasks  pertinent  to  specific 
occupational  areas.  In  completing  an  inventory  for  any  given  position  within  an 
occupational  area,  a  job  incumbent  or  knowledgeable  subject  matter  expert  (SME) 
checks  or  rates  each  task  according  to  its  application  or  relationship  to  the  position.  The 
rating  may  be  in  accordance  with  any  of  several  possible  rating  factors,  such  as  time 
spent  performing  the  task,  judged  importance  or  significance,  judged  performance 
difficulty,  criticality  for  training,  or  the  estimated  time  required  to  learn  to  perform. 

Because  the  data  for  individual  jobs  are  expressed  in  quantitative  terms,  the  results  can 
be  subjected  to  computerized  statistical  analyses.  The  Comprehensive  Occupational 
Data  Analysis  Program  (CODAP),  for  example,  is  a  set  of  computer  programs  used  to 
automate,  process,  organize,  and  report  occupational  data.  The  Levine  et  al.,  1983  study 
consistently  ranked  CODAP  highly  as  an  effective  and  practical  job  analysis  method.  The 
specific  organizational  purposes  for  which  CODAP  received  high  ratings  were:  job 
description,  job  classification,  job  evaluation,  job  design,  worker  training,  worker  mobility, 
efficiency/safety,  manpower/workforce  planning,  and  legal/quasilegal  requirements. 
CODAP  was  rated  most  practical  in  terms  of  occupational  versatility/suitability, 
standardization,  respondent/user  acceptability,  operational,  reliability,  and  quality  of 
outcome. 

CODAP  can  be  used  to  group  together  jobs  that  have  similar  tasks  or  task 
combinations,  resulting  in  job  families  that  can  be  used  for  a  number  of  purposes. 
Government  agencies  and  large  industrial  firms  have  used  job  inventories-CODAP 
extensively  to  develop  training  programs  and  performance  criteria,  as  well  as  validate 
employee  selection  programs,  job  classifications,  job  descriptions,  and  job  evaluations 
(Cascio,  1991).  The  ability  to  accurately  define  the  composition  and  content  of  existing 
jobs  in  the  United  States  Air  Force  is,  as  stated  by  Carpenter,  Giorgia,  and  McFarland 
(1975),  a  vital  prerequisite  to  the  effective  operation  of  the  Air  Force  personnel 
management  system.  CODAP  provides  this  capability  to  the  Air  Force  and  her  sister 
services.  In  addition  to  being  the  job  analysis  standard  for  the  United  States  and  many 
foreign  military  services,  several  universities,  industrial  organizations,  and  non-profit 
organizations  have  collected  occupational  data  using  job  inventories  and  have 
accomplished  their  analysis  using  the  CODAP  system  (Christal,  1974). 
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CODAP 


The  most  extensive  job  analysis  programs  were  developed  and  are  operated  by 
governmental  agencies.  While  operational  and  legal  requirements  have  fostered 
extensive  growth  of  utilization  of  job  analysis  in  industry,  most  of  the  larger  job  analysis 
research  and  development  programs  still  reside  within  the  government.  The  Air  Force 
research  into  the  occupational  analysis  field  began  in  1956  impacting  the  growing  field  of 
industrial  psychology.  Specifically,  this  research  led  to  the  development  of  the 
Comprehensive  Occupational  Analysis  Program  (CODAP).  Today,  the  CODAP  system 
continues  to  be  updated  and  enhanced  by  the  USAF  Armstrong  Laboratory  (Phalen, 
Mitchell,  &  Hand,  1992).  Currently  available  are  CODAP  programs  which  can  organize 
and  analyze  occupational  data  to  answer  most  of  the  conceivable  questions  posed  by 
managers  of  a  personnel  system.  It  is  primarily  for  this  reason  that  all  military  services  in 
the  United  States,  many  allied  military  services,  as  well  as  a  number  of  other  government 
agencies,  academic  institutions,  and  some  private  industries,  use  CODAP  in  human 
resource  management  (Page  &  Van  De  Voort  cited  in  Cascio,  1991;  Christal,  1974; 
Mitchell,  1984). 

The  evolution  of  CODAP  began  with  the  original  version  of  CODAP  simply 
reporting  the  time-ranked  job  descriptions  for  a  specified  group  of  individuals.  The 
second  version  introduced  hierarchical  clustering  which  aided  in  identifying  those  types  of 
jobs  which  actually  existed  as  opposed  to  those  traditionally  thought  to  exist.  This 
technique  provides  valuable  feedback  to  management  of  large  or  geographically 
separated  organizations. 

The  next  major  advance  in  CODAP  technology  was  incorporation  of  a  method  of 
integrating  the  priorities  of  supervisors.  These  priority  levels  dealt  with  occupational 
factors,  such  as  training  time  to  reach  proficiency,  consequences  of  inadequate 
performance,  or  the  criticality  of  immediate  performance.  By  collecting  these  ratings  from 
supervisors  and  processing  them  with  CODAP's  interrater  reliability  program  to  remove 
noncooperative  raters,  a  prioritized  list  can  be  produced  which  represents  the  opinion  of 
supervisors  surveyed.  Another  major  advance  was  the  ability  to  recategorize  and 
summarize  the  task-level  information  into  higher-level  modules  more  meaningful  to 
managers.  Application  of  this  technology  has  resulted  in  improved  curriculum  design  and 
validation. 

One  of  the  more  recent  advances  to  the  system  is  called  profile  analysis.  This 
enhancement  permits  the  hierarchical  clustering  of  people  or  jobs  based  upon  data  items 
of  interest.  Potential  applications  include  studies  of  job  satisfaction  profiles  across  jobs, 
clustering  jobs  to  identify  job-related  requirements,  clustering  supervisors  to  determine  if 
different  priority  policies  are  at  work,  and  clustering  tasks  into  efficient  jobs,  jobs  into 
efficient  specialties,  and  specialties  into  efficient  career  fields  during  job  description 
activities  (Driskill,  1992;  Gould,  Archer,  Filer,  Short,  &  Kavanagh,  1989;  Weber,  1991; 
Weismuller  &  Thew,  1979). 

CODAP  is  a  dynamic  system.  It  contains  over  100  computer  programs  with  the 
capability  of  running  on  several  computer  systems  including  Sperry,  CDC,  and  IBM 
compatible  computer  equipment.  The  current  package  has  the  ability  of  processing 
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20,000  cases  of  3,000  task  ratings  and  8,000  characters  of  background  information  per 
case  (Weber,  1991).  CODAP  technology  has  made  it  possible  to  study  jobs  on  the  scale 
necessary  to  work  with  the  most  critical  aspects  of  Air  Force  career  fields. 

Air  Force  Occupational  Measurement  Program 

The  first  Air  Force  operational  occupational  analysis  effort,  the  Job  Specialty 
Survey  Division,  was  formed  in  1967  to  apply  CODAP  technology.  Later  in  1972,  the 
organization  now  called  the  United  States  Air  Force  Occupational  Measurement 
Squadron  (USAFOMS)  was  formed  at  Lackland  AFB,  Texas.  USAFOMS  and  its 
occupational  analysis  program  relocated  to  its  current  location  at  Randolph  AFB,  Texas  in 
1978. 


USAFOMS  is  the  office  of  primary  responsibility  for  conducting  job  analysis  for 
Air  Force  jobs  worldwide.  The  occupational  survey  process  is  done  in  five  phases. 

Phase  one  is  development  of  the  job  survey  instrument  called  the  USAF  Job  Inventory. 
Inventory  developers  interview  subject  matter  experts  to  determine  the  tasks  performed  in 
an  Air  Force  Specialty  (AFS).  From  these  interviews  a  comprehensive  listing  is  obtained 
of  all  tasks  that  may  be  performed  by  career  ladder  or  job  incumbents.  This  task  listing, 
along  with  pertinent  background  questions  (job  title,  equipment  operated  or  maintained, 
training,  etc.)  is  then  published  as  a  USAF  Job  Inventory.  The  second  phase  consists  of 
administering  the  USAF  job  inventory  to  eligible  job  incumbents  to  collect  "percent 
members  performing,"  "percent  time  spent,"  and  "task  factor"  data.  Task  factor  data, 
including  Task  Difficulty  (TD)  ratings  and  Training  Emphasis  (TE)  ratings,  are  collected 
from  experienced  senior-level  technicians.  In  phase  three,  the  completed  booklets  are 
reviewed  and  quality  controlled  to  ensure  adequate  and  quality  data  are  being  collected. 
The  returned  inventories  are  optically  scanned  or  key-punched  by  data  processing 
personnel  into  the  computer.  In  the  fourth  phase,  data  are  analyzed  using  a  series  of 
CODAP  programs  and  the  results  are  reported  in  Occupational  Survey  Reports  (OSRs) 
which  summarize  findings  and  implications  on  enlisted  classification,  training,  and 
personnel  decisions.  In  phase  five,  USAFOMS  publishes  and  provides  OSRs  and  related 
data  packages  to  interested  Air  Force  agencies. 

The  airman  analysis  program  routinely  surveys  all  Air  Force  Specialities  (AFSs) 
an  average  of  every  five  years.  With  over  250  specialities,'  approximately  25  specialties 
or  over  70,000  airmen  are  surveyed  each  year  with  results  published  as  Occupational 
survey  reports  (OSRs).  A  typical  survey  booklet  contains  roughly  between  12  and  20 
duties  and  approximately  500  to  800  tasks  (Dubois,  1 992).  Background  items  vary 
among  AFS  surveys  depending  on  relevant  issues  effecting  the  particular  specialty. 
However  12  standard  questions  are  usually  always  included.  These  items  request  such 
information  as  paygrade,  major  command,  organizational  level,  total  active  military 
service,  time  in  job,  and  job  satisfaction. 

Task  inventories  satisfy  the  requirements  for  a  flexible,  economical,  reliable,  and 
valid  procedure  for  systematically  collecting  and  organizing  information  for  Air  Force  jobs 
(Morsh,  Madden,  &  Christal,  1961).  Such  job  analysis  information  serves  as  the  basis  for 
modification  of  the  existing  classification  structure  and  personnel  selection  techniques, 
measurement  of  job  difficulty  and  incumbent  performance,  and  forms  the  necessary  input 
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for  the  determination  of  appropriate  job  reengineering  actions,  as  well  as  identification  of 
training  requirements,  setting  of  aptitude  standards,  and  job  satisfaction  research 
(Carpenter  et  al.,  1975;  Christal,  1974). 

Task  difficulty  is  a  task  factor  routinely  collected  by  the  USAFOMS’s  occupational 
measurement  program.  It  plays  an  integral  role  in  planning  for  many  of  Air  Force’s 
programs,  such  as  in  developing  reassignment  systems,  designing  training  programs, 
determining  aptitude  requirements,  and  determining  grade  requirements  for  positions. 
Task  difficulty's  extensive  use,  despite  its  relatively  recent  development,  has  made  it  a 
critical  component  of  Air  Force  operational  and  research  programs. 

Task  Difficulty 

Utilizations 


The  information  derived  from  job  analysis  is  fundamental  to  management 
information  and  may  be  regarded  as  the  keystone  of  the  personnel  system  of  the  Air  force 
(Morsh  et  al.,  1961).  Possible  uses  for  job  analysis  results  in  the  USAF  are  listed  in 
Figure  3.  As  Morsh  et  al.  described,  such  information  is  essential  in  the  measurement  of 
job  proficiency,  in  selection  and  classification,  in  development  of  training  curricula,  in 
organizational  restructuring,  and  in  improvement  of  the  work  environment  and  job 
satisfaction. 

Task  difficulty  represents  a  relatively  small  but  critical  portion  of  job  analysis  in 
the  Air  Force.  Mead  (1970)  proposed  several  uses  of  task  difficulty  data.  Following  is  a 
list  of  possible  task  difficulty  applications. 

1 .  Assist  in  establishing  minimum  aptitude  requirements  for  positions  and 

classes  of  positions 

2.  Compare  the  difficulty  level  of  work  assigned  to  individuals  at  various  aptitude 

levels 

3.  Develop  reassignment  systems  that  would  insure  individuals  are 
assigned  jobs  of  increasing  difficulty  and  responsibility  as  they  progress 
in  their  career  ladders 

4.  Determine  appropriate  grade  requirements  for  positions 

5.  Compare  the  difficulty  level  of  work  assigned  to  technical  school  graduates, 

individuals  bypassing  technical  school,  and  individuals  receiving  Direct 
Duty  Assignments  (DDA). 

6.  Guide  decisions  about  modifications  in  classification  structure. 

7.  Investigate  the  interaction  between  job  difficulty,  job  satisfaction,  utilization, 

and  career  decisions. 


? 


8 


Uses  of  Job  Analysis  in  the  United  States  Air  Force 

1 .  To  identify  and  organize  job  content  for  writing  and  revising  standardized  specialty 
descriptions. 

2.  To  establish  objective  information  for  job  evaluation  and  to  guide  or  support  decisions 
concerning  grade  levels,  promotion,  upgrading,  transfer,  on-the-job  or  cross  training,  or 
reorganization  of  work  units. 

3.  To  determine  job  differences  and  job  relationships  to  be  used  in  structuring  and 
organizing  existing  Air  Force  jobs  into  specialties,  officer  utilization  fields,  and  airman 
career  fields. 

4.  To  identify  the  essential  features  of  jobs  for  the  inference  of  skills,  knowledges,  and 
other  personal  attributes  requisite  for  current  Air  Force  jobs  and  specialties  or  for 
projecting  requirements  of  new  tasks,  jobs,  or  specialties. 

5.  To  provide  detailed  information  about  current  specialties  which  can  be  used  to  identify 
changes  in  personnel  requirements  resulting  from  the  installation  of  new  equipment, 
tools,  or  methods  of  work. 

6.  To  contribute  to  efficient  utilization  of  individuals  and  effective  career  planning  and  to 
the  projection  of  the  future  manpower  requirements  of  the  Air  Force. 

7.  To  provide  essential  data  to  be  used  in  the  development  of  job  training  standards, 
course  training  standards,  or  for  the  revision  of  training  curricula. 

8.  To  provide  a  basis  for  validating  Qualitative  Personnel  Requirements  Information 
procedures. 

9.  To  improve  the  matching  of  men  with  jobs  which  will  aid  in  the  development  of  more 
effective  selection  and  classification  devices. 

10.  To  supply  current  information  for  the  construction  of  proficiency  tests  and 
establishment  of  work  performance  standards  by  determining  time  required,  methods 
used  in  performing  designated  units  of  work,  and  other  pertinent  data. 

11.  To  support  work  simplification  and  organizational  analysis  programs  designed  to 
improve  work  methods,  procedures,  or  equipment  furthering  efficiency  of  individual  units. 

12.  To  isolate  factors  which  induce  job  satisfaction,  raise  morale,  or  bring  about 
increased  effectiveness  of  individual  effort. 

13.  To  locate  health  and  accident  hazards  and  to  suggest  safety  precautions. 

Figure  3.  Uses  of  job  analysis  in  the  USAF 


Several  research  projects  have  incorporated  task  difficulty  data.  These  projects 
are  based  on  the  assumption  that  task  difficulty  is  based  on  task  learning  difficulty.  For 
example  in  1972  and  1973,  Fugill  successfully  hypothesized  relative  task  aptitude  (level 
of  aptitude  required  to  ensure  satisfactory  performance  of  a  given  task)  is  conceptually 
inseparable  from  relative  task  difficulty  when  difficulty  is  defined  in  terms  of  learning  time. 
In  the  early  1980's,  the  Air  Force  Human  Resources  Laboratory  (AFHRL)  developed  a 
method  to  determine  minimum  aptitude  standards  for  USAF  career  specialties  based  on 
task  difficulty  and  task  aptitude  benchmarked  rating  scales.  These  scales  provide  a 
common  reference  in  which  to  compare  tasks  from  different  AFS  aptitude  areas  (Burtch, 
Lipscomb,  &  Wissman,  1981;  Ramadge,  1987). 

An  individual's  cognitive  aptitude  for  entry  and  assignment  into  the  Air  Force  is 
measured  by  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB).  Composites  of  ASVAB  subtests  are  used  to  establish  a  person's  aptitude  for 
mechanical,  administrative,  general,  and  electronics  (MAGE)  occupational  areas  and 
percentile  scores  are  identified  for  qualification  into  a  specialty  job  (Ramadge,  1987).  The 
USAF  Airmen  Classification  Structure  classifies  each  specialty  by  cognitive  aptitude  area 
or  MAGE  type,  and  a  minimum  aptitude  requirement  is  assigned  to  each  AFS. 
Occupational  aptitude  requirements  are  critical  parameters  in  personnel  procurement, 
training,  utilization,  and  job  attitudes.  Prior  to  this  research  effort  (Burtch,  Lipscomb,  & 
Wissman,  1981;  Ramadge,  1987),  determination  of  appropriate  minimum  aptitude 
requirements  for  AF  specialties  was  primarily  a  function  of  recruiting  and  training 
objectives  and  was  established  and  modified  subjectively  and  unsystematically  (Christal, 
1974;  Fugill,  1972;  Weeks,  1984).  The  minimum  aptitude  requirement  for  entry  into  an 
AFS  is  now  established  by  the  Air  Force  Military  Personnel  Center  (AFMPC)  using  task 
factor  data  and  CODAP  methodologies  developed  by  the  AFHRL. 

AFHRL’s  systematic  procedure  for  inferring  relative  aptitude  requirements  is 
referred  to  as  occupational  learning  difficulty  (OLD).  OLD  is  defined  as  the  amount  of 
time  it  takes  to  learn  an  occupation  satisfactorily.  Three  types  of  occupational  information 
is  employed  in  deriving  OLD.  Two  of  the  components  are  task  difficulty  ratings  and  task 
time  spent  ratings  as  collected  from  USAFOMS.  The  third  element  involves  benchmark 
rating  scales  which  are  designed  to  capture  the  range  of  learning  difficulty  characteristics 
of  all  tasks  in  an  occupational  aptitude  area  (Davis,  1989;  Weeks,  1981, 1984). 
Benchmark  ratings  of  task  learning  difficulty  based  on  task-anchored  benchmark  rating 
scales  are  comparable  across  occupations.  An  OLD  index  computed  by  cross  multiplying 
resulting  benchmark  values  by  the  corresponding  percent  time  spent  value  yields  an 
average  task  difficulty  per  unit  time  spent  (ATDPUTS)  index,  which  are  then  aggregated 
across  all  tasks  in  the  occupation.  Thereby,  the  most  difficult  AF  specialties  should  have 
the  highest  minimum  aptitude  requirement .  Results  of  these  research  efforts  indicated 
that  aptitude  minimums  for  some  specialties  were  seriously  misaligned  (Weeks,  1981, 

1 984).  Thus  OLD  provided  the  frame  of  reference  for  determining  aptitude  minimums. 
While  previous  research  conducted  by  Mumford,  Weeks,  Harding,  and  Fleishman  (1987) 
provided  evidence  of  OLD’s  construct  validity,  further  research  into  assessing  OLD's 
construct  validity  is  currently  in  progress  (Dickenson,  1991). 

Another  series  of  studies  integrating  task  difficulty  data  relates  to  automated  test 
outlines  (ATOs)  developed  for  AF  specialty  knowledge  test  (SKT)  construction.  In  the  Air 
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Force,  promotion  to  the  E-5  through  E-7  enlisted  paygrades  in  most  specialties  is 
determined  by  an  airman's  relative  ranking  in  the  Weighted  Airman  Promotion  System 
(WAPS).  One  of  the  most  critical  factors  in  determining  promotion  success  is  SKT 
scores,  a  component  of  WAPS.  The  SKT  is  a  100-item  multiple  choice  test  designed  to 
measure  job  knowledge  of  an  AFS.  The  USAFOMS  Test  Development  Flight 
(USAFOMS/OMD)  is  responsible  for  development  of  SKTs  (Longmire,  Phalen, 

Weismuller  &  Dittmar,  1988).  Although  USAFOMS/OMD  has  maintained  a  successful 
test  development  program  since  the  1950's,  continuing  efforts  to  improve  and  enhance 
testing  procedures  led  to  a  large-scale  research  effort  to  develop  and  validate  a 
methodology  for  producing  occupational  data  based  outlines  for  SKTs  (Phalen,  Dittmar,  & 
Weismuller,  1989). 

The  first  step  in  the  ATO  process  is  to  predict  a  subset  of  USAF  Job  Inventory 
tasks  which  has  the  highest  relevance  for  testing  purposes.  A  standard  weighted 
regression  equation  which  considers  percent  members  performing,  percent  time  spent, 
training  emphasis,  and  task  difficulty  values  for  each  task  generates  a  predicted  testing 
importance  (PTI)  value  for  E-5  and  E-6/7  tasks.  These  subsets  of  tasks  are  administered 
to  an  average  of  50  to  70  noncommissioned  officers  (NCOs)  who  rate  each  task  on  its 
specialty  knowledge  testing  importance.  The  resulting  field  inputs  are  processed, 
analyzed,  and  subsequently  used  to  determine  testing  importance  weight  for  each  task 
and  to  calculate  test  outline  weights  for  each  major  area  (Weismuller,  Dittmar,  &  Phalen, 
1988).  Efforts  to  evaluate  the  ATO  process  examined  the  reliability  and  validity  of  the 
primary  components  using  field  validated  testing  importance  (FVTI).  Task  difficulty  data 
exhibited  a  stronger  relationship  with  FVTI  at  an  E-6/7  level  than  an  E-5  level.  This 
relationship  would  be  expected  if  task  difficulty  is  assumed  to  be  an  estimate  of  how 
difficult  it  is  to  learn  to  perform  a  task  and  senior  experienced  personnel  perform  those 
tasks  which  are  more  difficult  to  learn  (Weismuller  et  al.,  1988).  Further  research  by 
Dittmar,  Weismuller,  Haynes,  &  Phalen  (1989)  indicated  that  the  development  of  AFS 
specific  PTI  equations  might  improve  predictor  efficiency.  While  research  continues, 
USAFOMS/OMD  has  successfully  implemented  the  ATO  process  with  SKT  development 
teams  and  test  psychologists  showing  increasing  acceptance  of  the  procedures  and 
products  (Phalen,  Laskowski,  &  Williams,  1989). 

A  final  example  of  research  projects,  applications,  and  impact  of  task  difficulty 
data  in  the  operational  Air  Force  is  the  development  of  Automated  Training  Indicators 
(ATI).  AF  Regulation  8-13  requires  specialty  training  standards  (STSs)  be  used  to 
document  training  requirements  for  each  specialty  in  terms  of  tasks  required  in  the 
specialty  and  the  knowledge  and  proficiency  required  to  perform  the  tasks.  This 
document  helps  standardize  on-the-job  training  (OJT),  as  well  as  highlights  formal 
training  needed  as  individuals  progress  through  their  career.  A  course  training  decision 
table  provides  guidance  for  training  decisions,  such  as  development  of  STSs  and  other 
course  documents.  The  table  assists  training  personnel  in  determining  what  training  is 
needed  and  to  what  depth  it  should  be  taught  (Fodale  &  Aslett,  1987).  As  shown  in 
Figure  4,  the  course  training  decision  table  contains  five  columns.  The  first  column 
references  the  criterion  group  and  percent  members  performing  a  given  task.  The 
second  column  addresses  TE  ratings  and  the  third  column  addresses  TD  ratings.  High 
TD  ratings  suggest  tasks  which  are  difficult  to  learn  to  perform  warrant  consideration  for 
centralized  training.  Tasks  rated  low  in  difficulty  are  considered  simple  to  learn  to 
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perform  and  are  recommended  for  OJT  with  no  centralized  training  unless  TE  ratings  or 
percent  members  performing  data  were  high.  The  fourth  column  examines  safety  or 
critical  issues.  The  fifth  column  provides  the  resultant  training  decision.  While  training 
development  personnel  realized  the  utility  of  the  data,  this  tabular  method  was  considered 
to  be  labor  intensive.  The  need  for  a  better,  more  organized,  and  faster  approach  was  in 
demand  Fodale,  1988). 

Development  of  an  automated  version  of  the  decision  logic  table  resulted  in  ATI. 
ATI  are  CODAP  processed  numeric  values  derived  from  comparisons  of  percent 
members  performing,  TE,  and  TD  ratings  for  a  given  AFS  similar  to  the  logic  of  the  course 
training  decision  table.  An  ATI  value  of  one  through  18  categorizes  the  task  into  a  type  of 
training  ranging  from  ”1"  train  by  OJT  with  no  centralized  training  to  ’’18"  centralize 
training  with  task  knowledge  and  performance  base.  ATI  was  field  tested  by  USAFOMS 
analysts  and  training  personnel  with  favorable  results.  Use  of  ATI  is  now  standard  policy 
in  USAFOMS'  occupational  analysis  program  (Fodale,  1988;  Fodale  &  Aslett,  1987). 

The  Air  Force,  as  all  of  the  Department  of  Defense,  is  undergoing  a 
significant  drawdown.  The  new  world  environment  and  national  priorities  have  dictated  a 
reduction  in  military  manpower  (Boles,  1992).  As  the  numbers  of  persons  entering  the  Air 
Force  decrease  and  complexity  and  scope  of  the  jobs  increase,  it  becomes  exceedingly 
necessary  to  make  optimal  use  of  whatever  manpower  is  available.  Task  difficulty  serves 
as  an  important  tool  to  improve  airman's  career  planning,  the  assignment  system,  job 
grade  requirements,  and  analysis  of  work.  Task  difficulty  data  provide  information  about 
the  learning  burden  of  tasks  that  make  up  Air  Force  jobs  and  the  ability  needed  by  airmen 
who  perform  those  tasks.  For  personnel  research,  the  data  are  useful  in  studying  the 
relationship  between  job  satisfaction,  reenlistment  intent,  and  utilization  of  airmen's 
talents  and  training. 

Definitions 


Task  difficulty  has  been  defined  as  "the  amount  of  time  needed  to  learn  to  do  a 
task  satisfactorily."  This  definition,  as  well  as  task  difficulty  data  collection  and  analysis 
techniques,  is  the  result  of  a  lengthy  stream  of  research  dating  back  to  1960  (Tarteli, 
1990).  As  a  result,  it  is  promoted  by  the  Occupational  Measurement  Program  and  is  the 
basis  of  many  policies  and  practices  of  the  USAF,  such  as  those  described  in  the  above 
section. 
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AUTOMATED  TRAINING  INDICATOR  (ATI)  TABLE 


Percent 

Members 

Performing 

Training 

Emphasis 

Ratings 

Task 

Difficulty 

Ratings 

Additional  Task  Factors 

ATI 

49.5  to  100 

High 

3.0  to  9.0 

18 

0  to  2.99 

Documented  Critical  or  Safety  Item 

13 

49.5  to  100 

Above  Average 

3.0  to  9.0 

17 

0  to  2.99 

8 

49.5  to  100 

Mean  or  Below 

3.0  to  9.0 

Determine  why  TE  is  low 

16 

0  to  2.99 

6 

29.5  to  49.4 

High 

3.0  to  9.0 

Documented  Critical  or  Safety  Item 

12 

0  to  2.99 

Documented  Critical  or  Safety  Item 

10 

29.5  to  49.4 

Above  Average 

3.0  to  9.0 

— 

15 

0  to  2.99 

MM 

29.5  to  49.4 

Mean  or  Below 

3.0  to  9.0 

Determine  why  TE  is  low 

mm 

0  to  2.99 

0  to  29.4 

High 

3.0  to  9.0 

Documented  Critical  or  Safety  Item 

ii 

0  to  2.99 

Documented  Critical  or  Safety  Item 

0  to  29.4 

Above  Average 

3.0  to  9.0 

0  to  2.99 

— 

0  to  29.4 

Mean  or  Below 

3.0  to  9.0 

— 

0  to  2.99 

LU 

High  =  1  Standard  Deviation  or  more  above  the  mean 
Above  Average  =  Mean  to  1  Standard  Deviation  above  the  mean 


18  =  Task  Knowledge  and  Performance 
17  =  Task  Knowledge  and  Performance 

16  =  Task  Knowledge  and  Performance  (unless  otherwise  justified) 

15=  Task  Knowledge 

14  =  Task  Knowledge  (unless  otherwise  justified) 

13  =  Task  Knowledge  and  Performance  if  critical  or  safety  item;  if  not 
ATI  15  ifTD  >3.0 
ATI  8  ifTD  <3.0 

12  =  Task  Knowledge  and  Performance  if  critical  or  safety  item;  if  not 
AT1 15  ifTD  >3.0 
ATI  8  ifTD  <3.0 

11=  Task  Knowledge  and  Performance  if  critical  or  safety  item;  otherwise  OJT 
10  =  Task  Knowledge  if  critical  or  safety  item,  otherwise  OJT 
9  =  Task  Knowledge  if  critical  or  safety  item,  otherwise  OJT 
8  =  Train  by  OJT 

1-7=  No  Centralized  Training  _ 


Figure  4.  Course  Training  Decision  Table 
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Task  difficulty  is  a  complicated  concept  and  defining  it  is  not  a  simple  matter 
(Christal,  1974;  Madden,  1960).  A  task  may  be  characterized  as  difficult  because  it 
involves  one  or  more  of  the  following  conditions  (Madden,  1962). 

1.  The  incumbent  was  not  properly  trained  to  perform  the  task  and  this  training  is 
not  available  on  the  job. 

2.  Performance  standards  do  not  exist  or  are  not  clearly  defined. 

3.  Proper  tools  and  equipment  are  not  available  or  are  hard  to  obtain. 

4.  The  task  has  emotional  aspects  which  are  unpleasant  to  the  incumbent. 

5.  There  is  some  attribute  of  the  task  itself  which  makes  it  difficult.  It  may  require 
intense  concentration  or  vigorous  physical  exertion. 

6.  There  is  some  attribute  of  the  environment  in  which  the  task  is  performed 
which  makes  it  difficult.  There  may  be  excessive  noise  or  a  requirement  for 
extensive  cooperation  with  other  workers. 

7.  The  incumbent  does  not  have  the  ability  to  perform  the  task  easily  even 
though  other  workers  generally  consider  it  an  easy  task. 

Difficulty  as  a  task  attribute  may  be  defined  in  as  many  ways.  For  instance,  the 
definition  may  be  in  terms  of  the  amount  of  training  or  experience  required  to  perform  a 
task;  it  may  refer  to  mental  or  physical  effort;  or  difficulty  may  mean  complexity  or 
monotony,  or  something  else. 

It  appears  that  defining  difficulty  has  presented  a  general  problem  of  critical 
significance.  One  approach  to  dimensionality  of  difficulty  is  to  ask  incumbents  to  describe 
specific  tasks  that  are  difficult  and  then  to  explain  why  they  are  difficult.  This  method  was 
used  in  a  preliminary  study  conducted  by  Madden  (I960, 1962).  Madden  found  that 
when  incumbents  were  asked  to  list  tasks  which  they  found  difficult  to  perform  or  to  leam 
and  to  state  the  reasons  why  they  were  difficult,  reasons  given  could  be  classified  into  10 
separate  categories.  Following  are  the  ten  categories  which  resulted: 

1.  training 

2.  interpersonal  relations 

3.  frustration 

4.  pressure 

5.  attention 

6.  aptitude 

7.  working  conditions 

8.  forms 

9.  regulations,  technical  publications,  manuals 

10.  unclassified 
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Examples  of  reasons  tasks  were  categorized  as  difficult  due  to  "training"  included 
individuals  did  not  receive  any  training,  did  not  receive  enough  training,  or  changes  were 
occurring  so  rapidly  that  training  was  never  completed.  The  "interpersonal  relations" 
category  involved  reasons  associated  with  difficulty  in  receiving  or  giving  supervision, 
relations  with  peers,  and  coordination.  The  "frustration"  category  included  difficulty 
reasons  relating  to  lack  of  proper  tools,  not  enough  information,  unavailable  parts  or 
publications,  and  waiting.  Tasks  categorized  as  "pressure"  difficulty  included  reasons 
such  as  insufficient  time  to  do  the  work  properly,  responsibility  for  expensive  equipment, 
or  rapid  changes  that  require  a  great  deal  of  adaptability.  "Attention"  difficulty  type  tasks 
included  work  that  required  precision,  careful  naming  or  labeling,  or  was  very 
complicated.  Tasks  that  were  considered  difficult  due  to  "aptitudes"  were  a  result  of 
individuals  lacking  the  appropriate  abilities  or  skills  to  complete  the  task  properly.  An 
example  might  be  a  draftsman  who  is  unable  to  draw.  Climbing  ladders  or  performing 
dirty  work  were  examples  of  the  "working  conditions"  category.  "Forms"  described  the 
difficulties  associated  with  filling  them  out,  getting  information  for  them,  and  avoiding 
errors.  Difficulties  associated  with  "regulations,  technical  publications,  and  manuals" 
included  finding  them,  understanding  and  interpreting  them,  and  conflicting 
interpretations.  Finally,  the  "unclassified"  category  encompassed  irrelevant  statements 
that  were  not  considered  real  reasons  ("It's  hard  to  do."),  reasons  which  were  extremely 
unusual  and  unlikely  to  constitute  a  component  of  difficulty  having  operational 
implications  ("I  do  not  speak  English  well"),  and  statements  which  did  not  seem  to  fit  into 
any  category  in  which  there  were  at  least  one  other  statement. 

Nine  of  these  categories  represented  different  definitions  of  difficulty,  and  the 
majority  of  them  were  more  related  to  the  work  or  environmental  situation  in  general  than 
to  particular  tasks.  Basically,  the  "difficult  to  perform"  reasons  were  those  which  could 
appear  in  any  job  and  were  not  produced  as  a  result  of  any  peculiar  characteristic  of  a 
particular  job  or  task.  Similarly,  only  a  few  "difficult  to  learn"  reasons  could  be  directly 
identified  as  derived  from  the  properties  of  a  task.  For  instance,  insufficient  training, 
interpersonal  relations  with  supervisors  and  subordinates,  frustrating  factors  (waiting  for 
parts),  working  conditions,  aptitude  requirements,  and  completion  of  forms  constituted  a 
large  part  of  all  reasons  given.  Thus,  reported  difficulty  may  stem  from  environmental 
conditions,  personal  characteristics,  or  some  factor  inherent  in  the  task  itself  unless  the 
rating  scales  explicitly  directs  the  rater's  response  to  a  specific  domain  of  difficulty. 

One  way  of  using  the  difficulty  attribute  is  to  leave  it  undefined  so  the  incumbent 
will  identify  tasks  they  judge  to  be  difficult  about  their  jobs,  whatever  the  reason. 

Although  Ammerman  (cited  in  Morsh,  Madden,  &  Christal,  1961),  Cragun  &  McCormick 
(1967),  and  Madden  (1960, 1962)  did  this  with  a  small  degree  of  success,  difficulty  is 
often  perceived  differently  from  task  to  task  by  the  same  individual.  For  instance,  a  given 
incumbent  may  consider  task  "A"  to  be  difficult  for  reason  "X,"  but  task  "B"  may  be 
perceived  as  difficult  for  reason  "Y."  Furthermore,  interviews  of  job  incumbents  have 
revealed  that  the  same  task  may  be  considered  difficult  for  widely  varying  and  unrelated 
reasons  by  different  incumbents  (Madden,  1960, 1962).  Judgements  and  opinions 
collected  using  a  global  task  difficulty  concept  offers  little  about  the  type  and  rational 
behind  an  individual’s  rating.  This  information  can  be  misleading  and  has  minimal  value. 
Defining  the  dimensions  of  difficulty  clearly  and  accurately  and  collecting  judgements  and 
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opinions  regarding  specific  components  of  difficulty  provide  more  meaningful  information 
having  a  higher  utility. 

Although  many  definitions  were  considered,  task  difficulty  was  initially  defined  in 
terms  of  mental  difficulty  and  physical  difficulty  (McCormick  &  Tombrink,  1960).  As 
Madden’s  (I960,  1962)  research  predicted,  the  test-retest  reliability  was  relatively  low  and 
the  inter-rater  consistency  indicated  differences  existed  among  incumbents  in  their 
perceptions  of  the  difficulty  of  the  various  tasks. 

During  the  late  1960’s,  the  Air  Force  Occupational  Program  collected  data  using 
"Difficulty  of  Task  Performance"  and  "Difficulty  of  Learning  the  Task"  relative  rating  scales 
(Fugill,  1972;  Morsh  &  Archer,  1967;  Weeks,  1984).  However,  as  initially  discovered  by 
Madden  (1962),  senior-level  technicians  did  not  perceive  task  performance  difficulty  when 
defined  in  terms  of  the  difficulty  of  performing  a  task  satisfactorily  under  normal  conditions 
as  an  independent  task  property  but  rather  as  dependent  on  physical  working  conditions, 
experience,  and  interpersonal  relations.  As  a  result,  senior-level  technicians  did  not 
always  agree  on  the  relative  difficulty  of  a  given  task.  The  alternative  was  to  consider  a 
definition  which  reflects  the  amount  of  time  it  takes  for  individuals  to  learn  to  perform  a 
task  adequately.  Numerous  studies  demonstrated  that  senior-level  technicians  could 
achieve  high  levels  of  agreement  when  rating  task  on  learning  difficulty  (Cragun  & 
McCormick,  1967;  Lecznar,  1971;  Mead  1970a,  1970b;  Mead  &  Christal,  1970).  For 
example,  Christal  (1974)  found  that  while  supervisors  could  not  agree  on  a  time  it  takes 
for  workers  to  learn  to  perform  tasks;  supervisors  could  agree  that  if  other  factors  are  held 
constant,  workers  can  learn  to  perform  some  tasks  faster  or  slower  than  other  tasks. 

Thus,  task-learning  difficulty  was  defined  as  the  time  it  takes  to  learn  to  perform  a  task 
satisfactorily  (i.e.,  the  higher  the  learning  difficulty,  the  more  time  required  to  learn  to 
perform  the  task).  As  a  result,  this  definition  was  adopted  for  the  purpose  of  obtaining 
judgments  of  task  difficulty.  Weeks’  1981  study  further  supported  this  task  difficulty  ' 
definition  by  concluding  that  "knowledgeable  judges  can  reach  high  levels  of  agreement 
concerning  the  relative  learning  difficulty  of  work  tasks  when  learning  difficulty  is  defined 
in  terms  of  learning  time.”  Finally,  Burtch  et  al.  (1982),  again  provided  evidence  in 
support  of  the  reliability  and  validity  of  task-level  ratings  of  learning  difficulty.  The  USAF's 
current  definition  of  task  difficulty  as  "the  amount  of  time  needed  to  learn  to  do  a  task 
satisfactorily"  is  supported  by  this  20-year  stream  of  research. 

Data  Collection 


The  USAF  method  of  collecting,  analyzing,  and  reporting  task  difficulty  is 
relatively  complex.  The  essential  rationale  and  research  evidence  upon  which  the 
existing  USAFOMS  method  has  been  based  are  comprehensively  reported  by  Morsh  et 
al.  (1961).  Morsh  and  Archer  (1967)  set  forth  detailed  procedures  for  collecting, 
organizing,  analyzing,  and  reporting  information  describing  Air  Force  jobs.  Although  the 
analysis  routines  have  changed  significantly  during  the  past  30  years  and  still  continue  to 
evolve,  the  data  collection  procedures  have  remained  very  stable  (Phalen  et  al.,  1992). 

The  Air  Force  occupational  analysis  program  is  designed  specifically  for  large 
scale  administration  and  operational  application.  The  basic  data  in  the  identification  of 
tasks  come  from  survey  data  routinely  collected  at  USAFOMS.  Initially,  the  job  inventory 
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used  in  the  periodic  occupational  surveys  of  active  duty  jobs  are  developed  by  creating  a 
duty  outline  and  a  listing  of  task  statements  based  on  job  descriptions,  course  training 
standards,  and  other  published  materials.  Related  tasks  are  then  organized  within  duty 
categories  and  the  task  list  is  revised  based  on  work-site  observations  of  the  job  and 
input  from  technical  specialists.  When  finalized,  the  job  inventory  (Jl)  is  administered  to  a 
representative  sample  of  job  incumbents  within  a  specialty  to  collect  information  about  the 
relative  amount  of  work-time  spent  on  tasks  which  they  perform,  using  a  9-point  scale 
with  descriptive  bench  marks  ranging  from  "a  very  small  amount"  to  "a  very  large 
amount."  Specifically,  job  incumbents  are  asked  to  check  the  tasks  they  perform  in  their 
present  job  and  then  rate  those  tasks  in  terms  of  the  relative  amount  of  time  spent  on  that 
task.  Relative  time  spent  means  the  total  time  spent  doing  the  task  compared  with  time 
spent  on  each  of  the  other  tasks  performed  in  the  present  job.  These  data  are  compiled 
in  a  computer-generated  job  description  to  provide,  among  other  information,  an 
estimation  of  the  percentage  of  incumbents  who  perform  each  task  and  the  average 
percentage  of  time  spent  on  each  task  by  those  in  the  specialty  who  perform  it.  This 
same  information  can  be  reported  for  any  group  of  individuals  who  can  be  defined  by 
available  background  variables  such  as  Total  Active  Federal  Military  Service  (TAFMS), 
grade,  education,  and  time-in-job.  The  first  two  sections  of  a  job  inventory,  a  biographical 
section  and  a  background  section,  are  used  to  collect  general  information  about  the  job 
incumbent  and  their  job.  A  representative  example  of  items  collected  in  the  biographical 
and  background  information  data  collection  sections  is  located  in  Appendix  A. 

The  same  duty/task  list  is  administered  to  approximately  75  senior  NCOs,  usually 
supervisors,  who  are  asked  to  rate  the  tasks  on  training  emphasis  (TE),  based  on  how 
much  structured  training  is  required  for  first-term  personnel.  Structured  training  includes 
training  such  as  basic  resident  training,  formal  OJT,  and  first-term  career  development 
courses.  TE  raters  first  check  tasks  requiring  structured  training  and  then  they 
recommend  the  amount  of  first-term  training  emphasis  needed  based  on  a  9-point  scale 
ranging  from  "1"  indicating  extremely  low  training  emphasis  to  "9"  indicating  extremely 
high  training  emphasis.  TE  data  are  important  for  the  determination  of  initial  skills  training 
requirements  by  aiding  decisions  about  which  job  tasks  should  be  trained  and  to  what 
degree. 


Another  75  senior  NCOs  are  asked  to  rate  the  same  duty/task  list  on  task 
difficulty,  based  on  how  much  time  is  required  to  learn  the  tasks.  The  instructions  for 
completing  the  task  difficulty  inventories  are  located  in  Figure  5.  TD  raters  are  asked  to 
first  develop  a  frame  of  reference  for  rating  task  difficulty  by  scanning  the  entire  list  of 
tasks.  Then  they  are  requested  to  estimate  the  task  difficulty  ratings  for  each  task 
compared  with  other  tasks  in  the  inventory  on  a  9-point  scale  ranging  from  "1"  indicating 
extremely  low  difficulty  to  "9"  indicating  extremely  high  difficulty.  These  ratings  are  used 
to  compute  an  estimate  of  the  task  difficulty  of  each  task  compared  with  other  tasks  in  the 
inventory.  Christal  (1974),  Mead  and  Christal  (1970),  Ruck,  Thompson,  and  Stacy 
(1987),  and  Ruck,  Thompson,  and  Thompson  (1978)  determined  task  factor  values  could 
be  reliably  obtained  from  as  few  as  20  to  40  raters.  However,  USAFOMS  historical 
records  indicate  approximately  only  60  percent  of  task  factor  booklets  are  returned  from  a 
general  administration.  Accordingly,  task  difficulty  is  typically  administered  to  75  raters. 


INSTRUCTIONS  FOR  RATING  TASK  DIFFICULTY 

As  a  senior  technician,  you  have  been  selected  to  provide  needed  information  pertaining 
to  the  difficulty  of  tasks  performed  in  your  career  ladder.  This  information  will  be  of  value 
to  the  Air  Force  in  the  improvement  of  training,  classification,  and  testing  programs.  To 
accomplish  this  rating,  follow  the  procedure  listed  below. 

NOTE:  To  obtain  the  maximum  response  possible,  it  is  requested  that  you  rate  each  task 
of  which  you  have  any  knowledge.  Rate  those  tasks  you  presently  perform  or  supervise, 
those  tasks  which  you  have  performed  at  a  prior  time  in  your  career,  and  those  tasks 
which  you  have  observed  or  supervised  while  being  performed  by  others.  Most 
personnel  with  your  experience  and  background  will  be  able  to  rate  the  majority  of  the 
tasks  listed  and  in  many  cases  to  rate  all  of  them. 

STEP  1.  Develop  a  frame  of  reference  for  rating  task  difficulty.  For  this  survey,  task 
difficulty  is  defined  as  the  amount  of  time  needed  to  learn  to  do  each  task  satisfactorily. 
To  develop  a  frame  of  reference,  scan  the  entire  listing  of  tasks.  Pick  out  some  easy 
tasks  and  some  difficult  tasks.  Then,  find  some  tasks  which  fall  between  these  extremes 
that  are  of  average  difficulty.  Use  these  tasks  at  or  near  the  middle  of  the  range  as  a 
reference  point  for  judging  the  difficulty  of  all  tasks  in  the  inventory.  This  frame  of 
reference  will  be  used  for  completing  STEP  2. 

STEP  2.  Estimate  the  task  difficulty  rating  for  each  task  compared  with  other  tasks  in  this 
inventory.  Use  the  scale  shown  here  and  at  the  top  of  each  page  to  rate  each  task. 

1 .  Extremely  Low 

2.  Very  Low 

3.  Low 

4.  Below  Average 

5.  Average 

6.  Above  Average 

7.  High 

8.  Very  High 

9.  Extremely  High 

Begin  with  the  first  task  in  the  booklet  and  give  each  task  of  which  you  have  knowledge  a 
difficulty  rating  from  1  to  9;  record  the  value  opposite  the  task  statement  in  the  column 
titled  "TASK  DIFFICULTY."  Try  to  rate  every  task  on  each  page.  Remember  (from  STEP 
1)  that  you  are  comparing  each  task  with  the  other  tasks  in  the  career  field. 

STEP  3.  The  last  page  of  the  booklet  is  available  to  add  any  tasks  you  do  now  which  are 
not  listed.  Your  constructive  suggestions  in  improving  the  job  inventory  will  be  useful. 

STEP  4.  Review  the  booklet  to  see  that  you  have  rated  the  DIFFICULTY  of  all  tasks 
possible.  Each  task  can  be  given  only  one  rating. 

Figure  5.  Instructions  for  Rating  Task  Difficulty 
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Analysis 


These  data,  once  collected,  are  then  analyzed  using  the  Comprehensive 
Occupational  Data  Analysis  Program.  The  CODAP  programs  serve  as  the  basic 
analytical  technique  for  presenting  a  job  analyst  with  readily  interpretable  information  on 
the  content  of  Air  Force  jobs  and  specialties.  However,  the  validity  of  these  program's 
products  obviously  assume  and  require  accurate  input  data.  To  the  extent  that  a  subject 
matter  expert  cannot  provide  specific  reliable  information  concerning  the  correct  type  of 
difficulty,  errors  may  occur  in  the  resulting  recommendations. 

One  means  of  minimizing  the  effects  of  inaccurate  data  is  through  a  CODAP 
program  which  identifies  and  removes  divergent  raters.  A  divergent  rater  is  one  whose 
rating  behavior  demonstrates  that  the  rater  did  not  take  the  task  seriously  or  one  who 
unintentionally  rated  improperly,  such  as  one  who  reverses  the  meaning  of  the  rating 
scale.  Once  divergent  raters  are  identified,  they  are  examined  to  determine  if  there  are 
any  systematic  similarities  among  them.  Similarities  may  suggest  the  presence  of 
multiple  policies  in  the  AFS.  The  reliability  of  a  single  rater  is  viewed  for  the  expected 
correlation  between  that  randomly  chosen  rater  from  a  sample  and  another  rater 
randomly  chosen  from  the  same  sample.  The  reliability  of  a  composite  of  raters  is  the 
expected  correlation  between  the  mean  task  value  for  a  set  of  raters  drawn  from  the 
sample  and  the  average  task  means  of  an  equivalent  set  of  raters  drawn  at  random  from 
the  population  of  raters  from  which  the  sample  of  raters  was  drawn.  The  CODAP 
GRPREL  program  tells  how  many  raters  of  the  same  type  in  the  sample  would  be  needed 
to  achieve  a  reliability  of  a  composite  of  raters  that  would  equal  .90.  Therefore,  when 
reliability  of  a  composite  of  raters  equals  or  exceeds  .90,  it  is  determined  there  exists  high 
interrater  reliability  among  raters  (Christal  &  Weismuller,  1976;  Goody.  1976). 

Another  means  of  minimizing  the  effects  of  poor  data  involves  a  quality  review  of 
incoming  data.  Specifically  the  occupational  analysts  from  the  Airman  Analysis  Branch 
(USAFOMS/OMYO)  are  responsible  for  reviewing  at  a  minimum  10  percent  of  all 
returned  job  inventories  and  1 00  percent  of  all  task  factor  booklets.  This  quality  control 
procedure  is  one  method  of  ensuring  job  incumbents  are  following  instructions  for  filling 
out  the  booklets  and  that  the  data  look  realistic. 

The  occupational  analysts  are  also  responsible  for  analyzing  and  making 
recommendations  based  on  occupational  survey  data  for  the  enlisted  career  ladders.  Of 
primary  interest,  The  "1990  OMYO  Self-Inspection  Checklist"  located  in  the 
USAFOMS/OMYO  Occupational  Analyst  Handbook  (1990)  and  recreated  in  Figure  6, 
instructs  analysts  to  "Analyze  task  difficulty  data."  Section  4  of  the  Handbook, 

"Procedural  guide  for  writing  OSRs  for  Enlisted  AFSCs,"  contains  the  guidelines  for 
reviewing  task  difficulty  (TD)  data  when  writing  the  related  portions  of  the  OSR  .  These 
guidelines  provide  three  questions  to  be  asked  by  the  analyst  about  tasks  with  high  task 
difficulty  ratings:  What  are  the  tasks?;  Who  performs  the  tasks?;  and  Is  there  a  trend?. 
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The  Procedural  Guide  for  Writing  OSRs  for  Enlisted  AFSCs: 

TD  Survey  Data 

"Objectives:  To  be  able  to  provide  valuable  information  for  decision-makers  regarding 
training  decision,  we  must  first  gather  data  that  are  reliable  and  analyzable.  To  this  end, 
we  collect  secondary  task  factor  data  in  the  form  of  task  difficulty  and  training  emphasis. 
Each  of  these  collection  instruments  provide  very  specific  and  definitive  kinds  of  data. 
For  example,  task  difficulty  is  a  measure  of  how  long  it  takes  to  learn  how  to  do  a 
particular  task. 

"Analysis  Procedures:  The  primary  purpose  of  this  analysis  is  to  provide  information 
to  technical  training  center  personnel  which  may  be  used  to  review  and  update  current 
training  programs.  With  this  in  mind,  the  analyst  should  leave  no  stone  unturned  in 
his/her  search  for  answers  that  will  support  a  sound  training  policy.  It  is  incumbent  upon 
the  analyst  to  use  approved  statistical  techniques,  as  well  as  sound  judgement  in 
performing  the  analysis  process,  based  on  the  data  collected.  Once  the  analyst  has 
percent  performing  data,  task  factor  data,  and  properly  matched  the  STS  and  POI,  he/she 
can  then  compare  that  information  to  all  pertinent  documents." 

For  the  "Task  Difficulty"  subsection,  the  following  questions  should  be  addressed: 

1)  What  tasks  are  rated  highest  in  TD? 

2)  Are  the  highest  rated  tasks  performed  by  high  percentages 
of  first-term  airman,  7-skill  level  personnel,  or  both? 

3)  IS  there  a  pattern  found  for  tasks  rated  highest  in  TD? 

Figure  6.  Task  Difficulty  Analysis  " 

The  Problem 

The  utility  of  a  technique  to  determine  the  difficulty  level  of  Air  Force  jobs  based 
on  a  time  to  learn  definition  is  not  in  debate.  The  basic  question  is  whether  the  task 
difficulty  data  currently  collected  by  USAFOMS  is  in  fact  "time  it  takes  to  learn  to  perform 
a  task  satisfactorily"  or  are  raters  providing  different  interpretations  of  task  difficulty  in 
their  ratings? 

As  shown  in  Figure  5,  the  current  instructions  provided  to  the  NCOs  by 
USAFOMS  for  rating  task  difficulty  do  not  emphasize  the  "learning"  dimensionality  of  task 
difficulty.  The  instructions  state  the  definition  only  once  without  bold-face  or  underlining 
of  the  term  "learning"  to  highlight  or  draw  attention  to  this  important  distinction. 
Furthermore,  the  rating  scale  defined  at  the  top  of  each  task-rating  page  states  only  "task 
difficulty"  (see  Appendix  B).  When  task  difficulty  is  used  without  any  qualifiers, 
individuals  may  think  of  performance  difficulty  or  how  difficult  it  is  to  perform  the  task.  In 
which  case,  USAFOMS  may  be  essentially  providing  instructions  which  collect  task 
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"performance"  difficulty  or  a  "global"  difficulty  rather  than  "learning"  difficulty  ratings.  If  so, 
improper  guidance  may  be  given  to  training  developers  and  other  policy  makers.  One 
consequence  might  be  that  classification  personnel  establishing  high  aptitude 
requirements  for  specialties  which  have  tasks  that  are  very  hard  to  complete  or  perform 
but  whose  technical  learning  requirements  are  not  difficult.  While  interrater  agreement 
could  be  high,  it  only  means  that  raters  agreed  on  difficulty,  not  that  they  rated  only 
learning  difficulty.  Past  research  clearly  indicates  "time  to  learn"  is  a  valid  and  reliable 
means  of  collecting  and  understanding  task  difficulty  (Burtch  et  al.,  1982;  Christal,  1974; 
Lecznar,  1971;  Mead  1970a,  1970b;  Mead  &  Christal,  1970;  Weeks,  1981).  The  concept 
of  task  learning  difficulty  is  key  to  ensuring  proper  data  are  collected  from  the  raters.  The 
users  of  task  difficulty  information  base  decisions  with  the  assumption  the  data  measure 
the  task  learning  difficulty.  There  is  a  strong  need  to  know  if  task  difficulty  data  currently 
being  collected  are  actually  task  learning  difficulty,  and  if  not,  to  identify  what  was  being 
collected,  as  well  as  a  possible  method  to  ensure  future  task  learning  difficulty 
instructions  are  clearly  understood  and  accurately  reported  by  subject  matter  experts. 

If  the  ratings  are  not  "pure"  learning  difficulty  measures,  the  required  corrective 
actions  might  be  minimal.  Re-titling  the  cover  page  as  Task  Learning  Difficulty, 
highlighting  and  emphasizing  the  definition  and  instructions,  and  titling  the  scales  located 
at  the  top  of  each  page  as  'Task  Learning  Difficulty"  might  be  sufficient  changes. 

However,  these  changes  may  have  a  serious  impact  on  validity  and  reliability  of  both  past 
and  future  data  (Demetriades,  Knoll,  &  Boyce,  1990).  Research  is  required  to 
demonstrate  the  relationship  between  task  difficulty  data,  as  collected  by  USAFOMS,  and 
data  which  emphasize  the  learning  aspect  of  difficulty. 

Therefore,  this  study  investigated  a  new  data  collection  procedure  which  modified 
the  current  techniques  by  using  a  clearer  more  concise  cover  page  statement, 
instructions,  and  difficulty  rating  scale  headings  with  the  expectations  that: 

1)  Significant  differences  will  be  identified  between  different  rating  methods  for  the 
same  tasks.  Specifically,  there  will  be  a  significant  difference  between  task  learning 
difficulty  and  task  difficulty/task  performance  difficulty  ratings. 

2)  Task  learning  difficulty  data  will  have  greater  rater  reliability  because  of  a  more 
focused  definition.  That  is,  task  learning  difficulty  will  have  fewer  divergent  raters  as  well 
as  higher  intra-  and  inter-rater  reliability  than  the  other  two  rating  procedures. 

3)  Task  learning  difficulty  will  appear  more  valid  as  a  measure  of  learning  difficulty 
through  the  specificity  of  its  technique  and  relationship  to  other  task  data,  such  as  percent 
time  spent,  time  functions  (seniority  and  experience  of  high  grade,  longer  service  time, 
and  greater  time  in  career  field),  and  training  emphasis  of  tasks  performed  by  incumbents 
in  their  first  jobs. 
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METHOD 


The  method  used  to  investigate  the  reliability  and  validity  of  task  difficulty  data 
was  three-fold  and  similar  to  the  standard  procedures  currently  used  to  collect  task 
difficulty  data.  Three  equivalent  samples  of  members  in  an  Air  Force  Specialty  (AFS) 
were  surveyed  in  a  single  administration.  Each  rater  received  one  booklet,  either  the 
current  task  difficulty  (TD)  survey  booklet,  a  new  experimental  task  learning  difficulty 
(Tip)  survey  booklet,  or  an  experimental  task  performance  difficulty  (TPD)  booklet. 
Training  emphasis  (TE)  booklets  and  job  inventory  (Jl)  surveys  were  also  administered  to 
AFS  personnel  according  to  standard  USAFOMS  procedures.  Raters  had  approximately 
3  months  to  complete  their  survey  booklet.  As  necessary,  follow  up  telephone  interviews 
with  a  sample  of  raters  were  coordinated  to  obtain  subjective  estimates  of  rating  scale 
differences. 

Materials 


The  Air  Traffic  Control  career  ladder  was  selected  as  the  specialty  to  be  used  for 
this  study  for  three  reasons.  First,  the  job  inventory  was  in  the  final  stages  of  completion 
at  the  time  of  this  study's  initiation.  Second,  the  population  size  was  large  enough  for 
administration  of  the  additional  survey  booklets;  and  thirdly,  the  nature  of  the  Air  Traffic 
Control  job  was  considered  suitable  for  investigating  task  difficulty  with  results  being 
generalizable  to  other  AF  specialties.  The  Air  Force  Speciality  Code  (AFSC)  272X0  Air 
Traffic  Control  career  ladder  job  inventory  task  list,  dated  June  1992,  was  provided  to 
senior  NCOs.  The  Job  Inventory  was  prepared  by  an  inventory  developer  after  carefully 
reviewing  pertinent  documents,  such  as  previous  task  lists  and  training  documents.  This 
task  list  was  refined  and  validated  through  personal  interviews  with  34  subject-matter- 
experts  representing  five  operational  bases.  This  process  resulted  in  a  final  job  inventory 
containing  514  tasks  organized  under  10  duty  headings. 

Three  types  of  difficulty  inventory  booklets  were  constructed.  One  booklet 
received  a  brown  cover  page  with  the  standard  "Task  Difficulty"  title  and  contained  the 
current  instructions  and  scale  headings  (see  Appendix  B).  Another  inventory  used  a  pink 
cover  page  with  the  revised  title,  "Task  Learning  Difficulty,"  and  revised  instruction  page 
and  scale  headings  (see  Appendix  C).  The  third  inventory  used  a  purple  cover  page  with 
the  revised  title,  "Task  Performance  Difficulty,"  and  revised  instruction  page  and  scale 
headings  (see  Appendix  D).  All  three  difficulty  booklets  contained  a  duplicate  page  to 
assess  internal  consistency  or  intra-rater  reliability.  Page  1 1  (tasks  229  through  252)  was 
chosen  for  duplication  and  was  located  following  page  10  and  again  following  page  13  in 
each  task  factor  booklet  (see  Appendix  E). 

Subjects 


All  eligible  senior  noncommissioned  officers  holding  a  Duty  Air  Force  Specialty 
Code  (DAFSC)  27270  designation  were  identified  using  the  Uniform  Airman  Record 
(UAR),  provided  by  Brooks  Air  Force  Base  (AFB)  Armstrong  Laboratory.  The  UAR  is 
maintained  by  the  USAF  Military  Personnel  Center  (USAFMPC)  at  Randolph  AFB  Texas 
From  the  1 ,307  eligible  personnel,  three  lists  of  75  names  randomly  selected  by  the 
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computer  were  generated.  Each  selectee  received  either  the  brown  Task  Difficulty  (TD) 
inventory,  the  pink  Task  Learning  Difficulty  (TLD)  inventory,  or  the  purple  Task 
Performance  Difficulty  (TPD)  inventory.  As  standard  occupational  survey  procedure,  a 
list  of  75  senior  NCOs  were  also  computer  selected  to  receive  a  blue  training  emphasis 
(TE)  booklet.  The  job  incumbents  receiving  task  factor  booklets  equate  to  26  percent  of 
the  total  eligible  DAFSC  27270  population.  Standard  procedure  also  dictates  that  if  an 
Air  Force  specialty  has  a  population  of  less  than  3,000  personnel,  all  eligible  members  be 
surveyed.  In  situations  where  the  population  exceeds  3,000  individuals,  the  analyst, 
survey  developer,  and  other  key  personnel  determine  an  appropriate  sampling  size. 

Since  4,683  AFSC  272X0  personnel  were  identified  as  eligible  to  participate  in  the 
survey,  a  50  percent  sampling  was  considered  acceptable  based  on  historical  data  and 
previous  survey  experience.  Therefore,  2,248  eligible  AFSC  272X0  job  incumbents  were 
programmed  to  receive  the  standard  job  inventory.  The  inventories  and  task  factor 
booklets  were  mailed  to  airmen  representing  all  skill  levels  and  all  using  major  commands 
(MAJCOMs)  at  99  installations  worldwide. 

Distribution  of  the  inventories  to  job  incumbents  was  carried  out  between  July 
and  November  1992  using  the  standard  occupational  survey  arrangements  with  base 
Consolidated  Base  Personnel  Offices  (CBPOs)  hosting  the  airmen.  The  Occupational 
Survey  Control  Officer  (OSCO)  located  at  each  CBPO  is  responsible  for  receiving  survey 
booklets  from  USAFOMS,  administering  the  survey  to  job  incumbents,  collecting  the 
completed  surveys,  and  forwarding  the  booklets  to  USAFOMS. 

Analyses 

Figures  7  through  11  summarize  the  primary  statistical  hypothesis  tested.  First, 
intrarater  reliability  was  evaluated  using  a  test/retest  scenario.  The  duplicated  pages  in 
each  booklet  provided  the  data  for  part  of  this  analysis.  The  results  of  the  first 
administration  (or  first  time  page  1 1  was  completed)  was  predicted  to  correlate  highly  with 
the  second  administration  (or  the  second  time  the  page  was  completed)  in  the  three 
survey  booklets  (see  Figure  7).  The  object  of  this  effort  was  to  ensure  an  individual  used 
the  same  logic  in  identifying  difficult  tasks,  for  at  least  the  same  task.  Higher  intrarater 
reliability  was  expected  for  task  learning  difficulty  ratings  and  was  tested  by  comparing 
the  correlations  of  TD  duplicated  tasks,  TLD,  and  TPD  duplicated  task  correlations. 
Though,  the  short  time  between  "testing"  may  effect  the  results,  it  will  probably  err 
towards  increased  correlation  due  to  proximity  and  temporal  effects,  i.e.  Type  II  error. 
Next,  interrater  agreement  was  measured.  The  duplicated  page  (second  page  11)  Was 
first  removed  and  then  CODAP  GRPREL  program  was  used  to  refine  the  data  by 
identifying  and  deleting  divergent  raters  using  the  technique  reported  by  Goody  (1976). 
Differences  in  the  number  of  divergent  raters  were  examined  with  the  expectations  that 
task  learning  difficulty  will  have  fewer  divergent  raters.  After  refining  the  data,  the  degree 
of  interrater  agreement  between  the  raters  was  measured  using  the  intraclass  correlation 
technique  described  by  Lindquist  (1953).  Because  the  tasks  are  rated  relative  to  each 
other  rather  than  on  an  absolute  scale,  Christal  and  Weismuller's  (1976)  adjustment 
option  was  used  to  convert  each  rater’s  scores  to  a  common  mean  of  5.0  and  a  standard 
deviation  of  1.00  (Keeth,  1990).  According  to  Phalen  and  Albert  (1992),  adjusted  ratings 
should  be  used  when  the  raw  ratings  are  believed  to  represent  no  more  than  the  position 
of  a  task  in  the  scale  relative  to  another  with  respect  to  the  characteristic  in  question. 
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Further,  Phalen  and  Albert  determined  that  adjusted  ratings  should  be  used  when  the 
only  concern  is  agreement  among  raters  in  terms  of  ranking  the  tasks.  The  task  factor 
ratings  are  treated  as  relative  scales  by  raters,  so  analyses  were  conducted  using  the 
standardized  rank  order  scores. 


Hypothesis  Testing 

H0 :  pjdi  TD2  =  no  correlation  between  first  page  and  second  duplicate  page 
in  TD  inventories 

Ha:  PTD1  TD2  *  0;  significant  correlation  between  first  page  and  second 
duplicate  page  in  TD  inventories 

Ho :  PTLD1  TLD2  =  0;  no  correlation  between  first  page  and  second 
in  TLD  inventories 

Ha  :  PTLD1  TLD2  *  0l  significant  correlation  between  first  page  and 
duplicate  page  in  TLD  inventories 

duplicate  page 

second 

Ho :  PTPD1  TPD2  =  0;  no  correlation  between  first  page  and  second 
duplicate  page  in  TPD  inventories 

Ha :  PTPD1  TPD2  *  significant  correlation  between  first  page  and 
second  duplicate  page  in  TPD  inventories 

TD  =  Task  Difficulty 

TLD  =  Task  Learning  Difficulty 

TPD  =  Task  Performance  Difficulty 

Figure  7.  Hypotheses  I 


Comparisons  of  each  survey  was  then  made.  Differences  between  "task 
difficulty"  ahd  "task  learning  difficulty"  survey  data;  and  "task  performance  difficulty"  and 
"task  learning  difficulty"  survey  data;  and  "task  difficulty"  and  "task  performance  difficulty" 
survey  data  were  examined  by  comparing  tasks  within  duty  areas  and  across  the  total 
survey,  as  well  as  creating  a  simple  correlation  matrix  to  examine  both  raw  and 
standardized  rank  order  task  factor  data.  Due  to  the  common  perceptions  associated 
with  task  difficulty  ratings,  a  small  relationship  was  expected  between  the  data  for  the 
three  task  factor  surveys.  Hinkle,  Wiersma,  and  Jurs  (1 979)  offer  a  "Rule  of  Thumb  for 
interpreting  the  size  of  correlation  coefficients: 


.90  to  1.00  Very  high  positive  correlation 
.70  to  .90  High  positive  correlation 
.50  to  .70.  Moderate  positive  correlation 
.30  to  .50  Low  positive  correlation 
.00  to  .30  Little  if  any  correlation 

While  differences  were  expected  to  exist  between  the  three  task  factors  (see  Figure 
8),  greater  differences  were  predicted  between  task  difficulty  and  task  learning  difficulty 
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then  between  task  difficulty  and  task  performance  difficulty  data.  This  prediction  is  based 
on  the  assumption  that  the  known  dimensions,  as  researched  by  Madden  (1960, 1962), 
support  distinctions  between  learning  and  performance  difficulty.  The  lack  of  clarification 
in  defining  the  general  instructions  of  task  difficulty  would  incline  respondents  towards  the 
less  specific  performance  aspect  of  difficulty  when  rating  tasks  (Cragun  &  McCormick, 
1967;  Madden,  1960,1962;  Morsh,  Madden,  &  Christal,  1961).  Interviews  with  subject 
matter  experts  were  used  to  clarify  the  type  of  difficulty  respondents  may  have  applied  in 
rating  the  tasks. 


Hypothesis  Testing 

Hq  :  ptd  TLD  -  -7°;  hi9h  positive  correlation  between  TD  and  TLD 
Ha  :  pjQ  jld  <  -70;  moderate  to  no  correlation  between  TD  and  TLD 

Hq  ;  ptld  TPD  ^  -7°;  hi9h  positive  correlation  between  TLD  and  TPD 
Ha  :  ptld  TPD  <  -70>  moderate  to  no  correlation  between  TLD  and  TPD 

Hq  :  ptd  TPD  ^  -70;  high  positive  correlation  between  TD  and  TPD 
Ha :  ptd  TPD  <  -70;  moderate  to  no  correlation  between  TD  and  TPD 


Figure  8.  Hypotheses  II 


Concerning  validation,  the  most  practical  method  to  determine  if  data  from  a 
particular  booklet  format  did  in  fact  measure  the  learning  aspect  of  task  difficulty  was  to 
compare  it  to  some  criterion.  Since  no  truly  adequate  criterion  was  available,  a 
convergent  validation  paradigm  was  used.  Percent  time  spent  appeared  to  be  a  suitable 
surrogate  criterion  measure.  Percent  time  spent  (PTS)  is  a  relative  rating  of  the  amount 
of  time  spent  performing  a  task  compared  with  time  spent  on  the  other  tasks  performed  in 
a  job.  If  a  task  which  is  more  difficult  to  perform  also  takes  longer  to  perform,  a  stronger 
relationship  should  exist  between  TD  or  TPD  and  PTS  than  the  relationship  between  TLD 
and  PTS  (see  Figure  9). 

Although  research  is  limited  on  this  line  of  reasoning,  Madden  (1961)  found  a 
positive  relationship  (.82)  between  the  length  of  time  it  takes  to  perform  a  task  and 
difficulty  ratings.  More  recent  research  by  McCauley,  O'Leary,  and  Rheinstein  (1991) 
found  only  minimal  correlations  between  'task  difficulty'  and  time  spent  performing  (.25). 
The  difference  might  be  in  the  definition  assigned  to  task  difficulty.  While  Madden  did  not 
formally  provide  a  definition,  McCauley  et  al.  used  a  learning  aspect  in  their  rating  scale. 
Again,  in  light  of  the  non-conclusive  data,  the  difficulty  data  were  validated  using 
interviews  with  subject  matter  experts.  Another  suitable  surrogate  criterion  is  one  or  more 
aspects  of  time:  time  in  the  career  field,  total  active  military  service  (TAFMS),  grade  and 
skill  level.  Both  Dittmar,  Driskill,  and  Weismuller  (1987)  and  Ruck,  Thompson,  and  Stacy 
(1987)  found  that  more  experienced,  higher-grade-level,  and  senior  personnel  performed 
the  more  difficult  tasks.  Earlier  research  by  Lecznar  (1971)  also  found  a  respectable 
correlation  (.70)  between  mean  task  difficulty  ratings  and  mean  grade  level  of 
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incumbents.  Stated  differently,  for  individual  tasks  there  is  a  degree  of  correspondence 
between  grade  level  of  individuals  assigned  to  perform  the  tasks  and  the  difficulty  (i.e.  the 
time  to  learn  to  perform  satisfactorily)  of  the  tasks.  Higher-grade-level  tasks  take  longer 
to  learn  to  perform  than  lower-grade  tasks.  Therefore,  task  learning  difficulty  should  have 
a  higher  correlation  with  aspects  of  the  more  senior  time  factors  than  task  difficulty  and 
task  performance  difficulty  (see  Figure  10). 


Hypothesis  Testing 

Ho :  ptd  PTS 1 PTLD  PTS-  lower  correlation  between  TD  and  percent  time 
spent  performing  tasks  than  TLD  and  PTS 

Ha :  PTD  PTS  >  PTLD  PTSl  higher  correlation  between  TD  and  percent  time 
spent  performing  tasks  than  TLD  and  PTS 

Ho :  PTPD  PTS  1  PTLD  PTS’  l°wer  correlation  between  TPD  and  percent 
time  spent  performing  tasks  than  TLD  and  PTS 
Ha  :  PTPD  PTS  >  PTLD  PTS’  higher  correlation  between  TPD  and  percent 
time  spent  performing  tasks  than  TLD  and  PTS 

Ho  :  PTPD  PTS  1  PTD  PTS>  lower  correlation  between  TPD  and  percent  time 
spent  performing  tasks  than  TD  and  PTS 

Ha  :  PTPD  PTS  >  PTD  PTS-  higher  correlation  between  TPD  and  percent  time 
spent  performing  tasks  than  TD  and  PTS _ 

Figure  9.  Hypotheses  III 


Hypothesis  Testing 

Ho :  PTLD  TM  -  PTD  TM-  lower  correlation  between  TLD  and  senior  time 
than  TD  and  time  factors 

factors 

Ha  :  PTLD  TM  >  PTD  TM-  higher  correlation  between  TLD  and  senior  time 
than  TD  and  time  factors 

factors 

Ho :  PTLD  TM  t  PTPD  TM'  lower  correlation  between  TLD  and  senior  time 
than  TPD  and  time  factors 

factors 

Ha  :  PTLD  TM  >  PTPD  TM!  higher  correlation  between  TLD  and  senior  time 
than  TPD  and  time  factors 

factors 

Ho  :  PTD  TM  =  PTPD  TM!  no  differences  between  TD  and  senior  time  factor 
correlations  and  TPD  and  time  factors 

Ha  PTD  TM  *■  PTPD  TM!s'9n'^cant  differences  between  TD  and  senior 
factor  correlations  and  TPD  and  time  factors 

time 

Figure  10.  Hypotheses  IV 
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The  final  criterion  to  be  examined  was  training  emphasis.  Ruck  et  al.  (1987) 
found  task  learning  difficulty  factors  had  significant  negative  correlations  with  training 
emphasis,  percent  members  performing  first  jobs  (1-24  months  TAFMS),  and  first  job 
percent  time  spent.  Negative  correlations  between  task  learning  difficulty  and  training 
emphasis  are  reasonable  to  expect  because  tasks  recommended  for  training  emphasis 
for  first  termers  (1-48  months  TAFMS)  should  be  those  tasks  that  they  routinely  perform 
during  their  first  jobs,  not  the  more  difficult  tasks  in  the  specialty.  Task  learning  difficulty 
should  then  have  higher  negative  correlations  with  training  emphasis  for  tasks  performed 
by  incumbents  in  their  first  jobs  than  task  difficulty  or  task  performance  difficulty  (see 
Figure  11). 


Hypothesis  Testing 

Hq  :  pjLD  TE 1 PTD  TE-  lower  (negative)  correlation  between  TLD  and  first  job 
training  emphasis  than  TD  and  TE 

Ha :  pjlq  te  >  PTD  TE'<  higher  (negative)  correlation  between  TLD  and  first  job 
training  emphasis  than  TD  and  TE 

Hq  :  PJLD  TE  —  PTPD  TE>  lower  (negative)  correlation  between  TLD  and  first  job 
training  emphasis  than  TPD  and  TE 

Ha  :  pjld  TE  >  PTPD  TE'  higher  (negative)  correlation  between  TLD  and  first  job 
training  emphasis  than  TPD  and  TE 

Hq  :  ptd  TE  =  PTPD  TE! no  differences  between  TD  and  first  job  training 
emphasis  correlations  and  TPD  and  TE 

Ha :  ptd  TE  *  PTPD  TE'  significant  differences  between  TD  and  first  job  training 

emphasis  correlations  and  TPD  and  TE _ 

Figure  1 1 .  Hypotheses  V 
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RESULTS 


Survey  Response 

The  survey  response  rate  for  the  job  inventory  and  task  factor  booklets  is 
presented  in  Table  1. 


Table  1.  Survey  Response  Rate 


BOOKLET 

MAILED 

RETURNED 

USABLE 

PERCENT 

OF 

MAILED 

Job  Inventory  (Jl) 

2,223 

1,859 

1,637 

74% 

Task  Difficulty  (TD) 

75 

60 

52 

69% 

Task  Learning  Difficulty  (TLD) 

75 

59 

53 

71% 

Task  Performance  Difficulty  (TPD) 

75 

63 

54 

72% 

Training  Emphasis  (TE) 

75 

62 

56 

75% 

As  illustrated  in  Table  2,  the  survey  sample  distribution  is  representative  of  the 
assigned  and  eligible  population  across  skill  levels.  The  1 ,637  respondents  represent  35 
percent  of  the  eligible  and  31  percent  of  the  assigned  AFSC  272X0  personnel. 


Table  2.  Skill  Level  Representation  of  Job  Inventory  Sample 


SKILL 

LEVEL 

ASSIGNED  a 

Total  Percent  of 

Assigned  Assigned 

ELIGIBLE  b 

Total  Percent  of 
Eligible  Eligible 

Jl  SAMPLE 

Total  Percent  of 
Sample  Sample 

| 

1,046 

20% 

1,002 

21% 

343 

21% 

1 

2,653 

51% 

2,308 

49% 

859 

52% 

27270 

1,299 

25% 

1,168 

25% 

383 

23% 

27290 

152 

3% 

140 

3% 

37 

2% 

27200 

70 

1% 

65 

1% 

15 

1% 

Total 

5,220 

100% 

4,683 

99% 

1,637 

99% 

Note.  Columns  due  not  add  to  100  percent  due  to  rounding. 


a  Assigned  Strength  as  of  March  1992 

b  Excludes  those  in  PCS,  retirement,  discharge,  or  hospital  status;  and  those 
with  less  than  6  weeks  on  the  job 


28 


Several  critical  biographical  items  were  used  to  identify  the  groups  needed  for 
analyses.  The  survey  sample  response  rates  for  these  key  groups  are  also  displayed  in 
Table  3.  As  shown,  response  rates  between  task  factor  booklets  are  highly  similar  across 
the  four  groups:  paygrade,  skill  level,  Total  Active  Federal  Military  Service  (TAFMS),  and 
Time  in  Career  Field  (TICF). 

Task  Factor  Reliability 

Hypothesis  I:  Testing  for  Intrarater  Reliability 

A  Pearson  product  moment  correlation  was  used  to  determine  the  relationship 
between  the  same  set  of  tasks  which  were  purposely  placed  in  the  booklets  twice.  Table 
4  lists  the  duplicate  tasks  and  their  corresponding  correlation  coefficient  for  each  task 
factor.  The  correlations  were  significant  atj><.05  with  most  at ^  <  .0001 .  The  one 
exception,  Task  F250  as  rated  with  task  performance  difficulty  (TPD)  guidance,  was 
recalculated  after  removing  two  divergent  raters,  Case  Control  Numbers  (CCNs)  18  and 
60 ,  which  increased  the  correlation  to  r  =.7727.  A  Fisher’s  z-transformation  at  .05  level 
of  significance  was  used  to  determine  if  correlations  were  significant.  All  duplicate  tasks, 
regardless  of  method  of  task  factor  rating,  showed  a  correlation  of  greater  than  or  equal  to 
.70.  Using  a  Student's  t-test,  differences  were  also  noted  in  the  mean  correlation  for  the 
24  duplicate  tasks.  At  a  .05  level  of  significance,  TPD  correlations  were  lower  than  TD 
and  TLD  correlations  (Mjq  =  0.816,  Mjlq  =0.829,  Mjpp=  0.738;  tQV  =  2.02,  df  =  46). 
Distribution  of  the  correlations  for  each  task  factor  identified  22  task  learning  difficulty 
(TLD)  tasks  with  correlations  greater  than  .70  versus  21  and  15  tasks  with  an  r  >  .70  for 
task  difficulty  (TD)  and  TPD  tasks  respectively.  Hinkle,  Wiersma,  and  Jurs  (1979) 
identified  correlations  less  than  .70  as  moderate,  low,  or  little  as  r  approached  zero. 

In  order  to  obtain  a  minimum  of  20  ratings  per  task,  survey  respondents  were 
asked  to  rate  all  tasks  in  which  they  had  knowledge,  including  tasks  they  currently  or 
previously  performed  and  those  tasks  they  observed  or  supervised  others  performing.  In 
some  cases,  respondents  are  unable  to  rate  every  task  due  to  lack  of  task  knowledge  or 
perhaps  lack  of  comfort  with  the  rating  scheme.  While  all  tasks  were  rated  by  more  than 
20  raters,  21  tasks  received  ratings  from  over  30  TLD  respondents.  Similarly,  19  tasks 
received  ratings  by  over  30  TPD  raters,  but  only  five  tasks  were  rated  by  more  than  30 
TD  raters.  A  comparison  of  the  mean  number  of  raters  for  tasks  229  through  252 
identified  the  only  significant  difference  fe  <  .05)  was  between  the  mean  number  of  TD 
raters  and  TLD  raters,  MTD  =  28.33,  Mjlq  =  32.38,  (tcv  =  2.02,  df  =  46).  The  mean 
number  of  TPD  raters  for  the  24  duplicate  tasks  was  30.13  and  was  not  significantly 
different  from  the  number  of  TD  or  TLD  raters. 
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Table  3.  Survey  Response  Rate  by  Critical  Groups 


GROUP 

Jl 

TD 

TLD 

TPD 

TE 

SAMPLE 

SAMPLE 

SAMPLE 

SAMPLE 

SAMPLE 

PAYGRADE 

E-1/2/3 

276 

E-4/5/6 

1,158 

34 

38 

36 

31 

E-7/8/9 

203 

18 

15 

18 

25 

SKILL  LEVEL 

27230 

343 

27250 

859 

1 

27270 

383 

51 

51 

51 

56 

27290 

37 

1 

1 

2 

27200 

15 

1 

TAFMS  (months) 

1-48 

473 

1-24 

162 

25-48 

311 

1 

49-96 

419 

97  + 

745 

52 

53 

54 

56 

97-144 

287 

7 

5 

3 

6 

145-192 

216 

22 

21 

20 

19 

193-240 

182 

19 

23 

29 

27 

241  + 

60 

4 

4 

.2 

4 

TICF  (months) 

1-24 

356 

1 

1-48 

722 

1 

49-96 

325 

1 

2 

1 

3 

97  + 

590 

51 

50 

53 

53 

Total 

1,637 

52 

53 

54 

56 
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Table  4.  Hypothesis  I:  Duplicate  Task  Correlations  within  Task  Factor  Booklets 


Duplicate 

Task 

TD 

r 

n  ; 

TLD 

r 

n 

TPD 

r 

n 

F229 

.666 

35 

.747 

37 

.783 

39 

F230 

.798 

28 

.873 

31 

.755 

32 

F231 

.768 

33 

.855 

34 

.549 

35 

F232 

.658 

31 

.817 

34 

.673 

34 

F233 

.669 

31 

.814 

33 

.828 

34 

F234 

.793 

31 

.898 

34 

.590 

33 

F235 

.812 

30 

.904 

34 

.789 

33 

F236 

.856 

30 

.862 

34 

.774 

32 

F237 

.721 

30 

.852 

34 

.760 

31 

F238 

.715 

30 

.835 

34 

.748 

32 

F239 

.890 

29 

.810 

33 

.767 

32 

F240 

.874 

29 

.865 

34 

.919 

33 

F241 

.848 

29 

.755 

33 

.739 

32 

F242 

.881 

29 

.813 

34 

.856 

32 

F243 

.845 

29 

.800 

34 

.861 

32 

F244 

.872 

18 

.573 

21 

.518 

23 

F245 

.856 

18 

.843 

25 

.601 

23 

F246 

.761 

28 

.880 

31 

.779 

32 

F247 

.880 

26 

.930 

33 

.805 

27 

F248 

.899 

28 

.821 

33 

.623 

33 

F249 

.912 

25 

.870 

29 

.694 

30 

F250 

.872 

29 

.902 

33 

.262a 

31 

F251 

.866 

25 

.951 

31 

.663 

29 

F252 

.872 

29 

.615 

34 

.858 

32 

£=.0001* 

a  corrected  to  r  =.773 


*  TLD  F244  £  = 
.0066 

TPD  F231  £  = 
.0006 

TPD  F234  £  = 
.0003 


TPD  F244  £  = 
.0114 

TPD  F245  £  = 
.0024 

TPD  F250  £  = 
.1545 
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Interrater  Reliability 


Similar  to  CCNs  18  and  60,19  task  factor  raters  were  identified  as  unacceptable 
by  the  CODAP  GRPREL  program.  As  with  most  studies,  reasons  raters  were  deemed 
as  divergent  include  a  high  percentage  of  unrated  tasks,  a  low  standard  deviation  from 
the  same  ratings  being  applied  to  all  tasks,  reversal  of  rating  scale,  or  general  non¬ 
cooperation.  As  depicted  in  Table  5,  six  raters  were  removed  from  the  task  difficulty 
sample,  one  rater  from  the  task  learning  difficulty  sample,  and  four  raters  were  removed 
from  the  task  performance  difficulty  sample.  Removal  of  these  raters  raised  the  reliability 
of  composite  raters  to  over  .90  which  is  considered  a  high  interrater  reliability  (Christal  & 
Weismuller,  1976;  Goody,  1976).  Using  a  normal  z-test  to  test  differences  between  two 
independent  proportions,  the  one  TLD  divergent  rater  of  the  sample  of  53  was 
significantly  (j>  <  .05;  zcv  =  1.96)  less  than  the  six  TD  divergent  raters  removed  from  the 
sample  of  52  TD  raters. 

Using  a  student's  t-test  (t^  =  1 .98,  i  <  .05,  df  =  1 004),  no  differences  were  noted 
between  the  mean  number  of  raters  per  task.  However  several  tasks  had  less  than  20 
raters,  specifically,  67  TD  tasks,  25  TLD  tasks,  and  50  TPD  tasks. 


Table  5.  Descriptive  Statistics  for  Raw  and  Standardized  Task  Factors 


TASK 

FACTOR 

N 

RAW 

M 

SD 

DIVERGENT 

RATERS 

N 

STANDARDIZED 

M  SD  R(k,k) 

MEAN 

RATERS/TASK 

TD 

52 

5.00 

0.88 

6 

46 

5.00 

1.00 

.941 

34.24 

TLD 

53 

5.26 

0.91 

1 

52 

5.00 

1.00 

.955 

43.16 

TPD 

54 

4.94 

1.11 

4 

50 

5.00 

1.00 

.960 

38.04 

TE 

56 

3.22 

2.27 

8 

48 

N/A 

N/A 

.974 

48.00 

Task  Factor  Ratings 

Hypothesis  II:  Testing  for  Relationships  Between  Task  Factor  Ratings 

The  remaining  correlation  analysis  was  completed  using  the  CODAP  Curves 
program  which  utilizes  the  Spearman  Brown  Prophecy  formula  for  rank  order  data.  The 
task  factor  correlation  matrix  with  corresponding  regression  values  are  displayed  in  Table 
6.  A  student's  t-test  (tcV  =  1 .684,  df  =  44)  at  a  .05  level  of  significance  was  used  to  test 
the  null  hypothesis  that  correlations  would  be  >  .70.  The  correlations  for  the  task  factors 
ratings  were  greater  than  .70.  Therefore,  the  null  hypothesis  could  not  be  rejected. 
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Table  6.  Hypothesis  II:  Task  Factor  Correlation  Matrix  and  Computed  R2  Values 


Raw/Standardized 

r 

R2 

TASK  FACTOR 

TD  TLD  TPD 

TD  TLD  TPD 

TD 

- 

- 

TLD 

.8939 

.7990 

TPD 

.8769  .9492 

.7689  .9009 

A  significant  relationship  was  found  using  Fisher's  z-transformation  in  testing  for 
differences  in  independent  correlations  between  tjlq  TPD  and  r  -j-q  jpq  .  Specifically 
the  .9492  correlation  between  TLD  and  TPD  is  significantly  higher  than  the  TD  and  TPD 
correlation  of  .8769  at  a  .05  level  of  significance  (zcv  =  1.96). 

Task  Factor  Rating  Distributions 

The  distribution  of  tasks  for  each  task  factor  is  presented  in  Table  7.  Of  the  51 4 
tasks,  79  tasks  were  rated  high  (M+1SD)  by  task  difficulty  raters.  Another  345  tasks  fell 
within  or  slightly  below  the  mean.  Similarly,  76  tasks  received  high  task  learning  difficulty 
ratings  and  355  rated  average  to  slightly  below  average.  Task  performance  difficulty 
raters  rated  78  tasks  as  very  difficult  with  343  falling  within  the  mean  or  1  standard 
deviation  below. 


Table  7.  Distribution  of  Task  Factor  Tasks 


FACTOR 

M+2SD 

M+1SD 

M 

M-1SD 

M-2SD 

M-3+SD 

TD 

mm 

75 

E3 

159 

59 

20 

TLD 

63 

H 

173 

58 

14 

TPD 

■9 

65 

177 

166 

66 

16 

Task  Data  Comparisons  Across  Survey  Booklet 

Tables  8,  9  and  10  list  the  tasks  receiving  high  (M+1SD)  difficulty  ratings  for  Task 
Difficulty,  Task  Learning  Difficulty,  and  Task  Performance  Difficulty  respectively.  A  core 
of  43  tasks  were  rated  as  highly  difficult  by  all  three  rating  strategies.  The  tasks 
considered  difficult  by  only  one  or  two  rating  methods  are  highlighted.  A  complete  listing 
of  AFSC  272X0  duty  titles  and  task  statements  are  located  in  Appendix  F. 


Table  8.  All  Task  Difficulty  Tasks  >  Mean  +  1  SD 


Table  9.  All  Task  Learning  Difficulty  Tasks  >  Mean  +  1  SD 


TASK  LEARI 

MING  DIFFICULTY 

TASK 

M 

TASK 

M 

TASK 

M 

TASK 

M 

C131 

7.55 

J493 

6.79 

F211 

6.81 

B  94 

6.43 

B  80 

7.52 

1446 

6.74 

C110 

6.80 

D161 

6.11 

J496 

7.34 

1445 

6.68 

D176 

6.43 

Cl  05 

6.10 

J499 

7.32 

B  55 

6.67 

A  37 

6.43 

J490 

6.10 

D154 

7.32 

J485 

6.66 

C130 

6.42 

J484 

6.08 

J500 

7.28 

J482 

6.64 

E188 

6.38 

A  16 

6.08 

J497 

7.19 

J480 

6.60 

B  64 

6.38 

El  86 

6.05 

B  62 

7.19 

1442 

6.60 

Cl  12 

6.36 

E201 

6.05 

J498 

7.14 

A  9 

6.60 

C133 

6.34 

J481 

6.04 

J495 

7.13 

J488 

6.59 

C  98 

6.33 

C104 

6.04 

A  8 

7.10 

J487 

6.58 

C135 

6.31 

1469 

6.04 

B  59 

7.10 

J478 

6.58 

Cl  32 

6.28 

1461 

6.03 

J494 

7.08 

B  61 

6.58 

J483 

6.21 

D165 

6.03 

B  81 

6.97 

J486 

6.57 

1452 

6.17 

1451 

6.02 

El  94 

6.97 

1472 

6.57 

El  99 

6.14 

F247 

6.02 

cm 

6.90 

1474 

6.57 

A  39 

6.14 

1475 

6.01 

J502 

6.85 

J479 

6.55 

D158 

6.14 

G362 

6.01 

A  15 

6.83 

J501 

6.48 

B  65 

6.12 

1471 

6.81 

El  98 

6.48 

J503 

6.44 

*  Bold  Tasks:  Tasks  not  rated  as  "high  difficulty"  by  TD,  TLD 
(Non-core  Tasks) 


and 


PD  raters 
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Table  10.  All  Task  Performance  Difficulty  Tasks  >  Mean  +  1  SD 


TASK  PERFORMANCE  DIFFICULTY 

TASK 

M 

TASK 

M 

TASK 

M 

TASK 

M 

J497 

7.39 

J502 

6.84 

B  59 

6.44 

El  86 

6.16 

J498 

7.27 

J485 

6.83 

1445 

6.43 

B  91 

6.16 

J495 

7.23 

C135 

6.77 

F211 

6.43 

A  18 

6.16 

J488 

7.19 

J479 

6.76 

D176 

6.42 

D143 

6.15 

J496 

7.19 

J490 

6.76 

A  16 

6.40 

1468 

6.13 

J500 

7.19 

A  8 

6.76 

C132 

6.37 

D158 

6.11 

J494 

7.10 

J482 

6.73 

1469 

6.37 

1475 

6.10 

B  94 

7.09 

J483 

6.72 

Cl  33 

6.35 

B  87 

6.09 

J486 

7.05 

D154 

6.69 

A  29 

6.35 

Cl  07 

6.09 

J499 

7.04 

A  15 

6.63 

B  55 

6.35 

F247 

6.08 

C131 

7.04 

J480 

6.60 

B  81 

6.34 

El  98 

6.08 

J492 

7.02 

J478 

6.58 

C110 

6.33 

B  66 

6.05 

J493 

7.02 

Cl  34 

6.57 

El  94 

6.32 

1451 

6.03 

J501 

6.99 

J484 

6.53 

C112 

6.32 

El  88 

6.02 

B  80 

6.99 

1472 

6.53 

C  98 

6.31 

B  74 

6.02 

J503 

6.94 

1446 

6.48 

B  61 

6.31 

A  20 

6.02 

1471 

6.90 

B  64 

6.47 

J481 

6.28 

A  7 

6.02 

J491 

6.88 

cm 

6.46 

B  89 

6.23 

G358 

6.01 

J489 

6.88 

B  62 

6.46 

A  39 

6.18 

J487 

6.85 

Cl  30 

6.45 

F265 

6.18 

*  Bold  Tasks:  Tasks  not  rated  as  "high  difficulty"  by  TD,  TLD  and 

TPD  raters 

(Non-core  Tasks) 

Task  Data  Comparisons  Within  Duty  Areas 

Survey  data  were  also  divided  into  duty  areas.  Tables  11,12,  and  13  display  task 
factor  information  across  the  duty  areas.  Specifically,  Table  1 1  includes  the  interrater 
reliability  of  ratings  for  each  duty.  The  task  factor  interrater  reliability  for  the  Duties  B  and 
F  are  high  for  the  three  task  factors.  Duties  A,  B,  D,  and  F  had  high  TLD  and  TPD  interrater 
reliability.  Duty  E  had  high  TLD  reliability.  The  average  rater  reliability  for  the  remaining 
duty  areas  were  acceptable,  except  for  TD  and  TPD  Duties  I  and  J  and  TLD  Duty  J.  Duty 
Area  J  also  averaged  less  than  20  raters  per  task  for  all  three  task  factors. 
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Table  1 1 .  Interrater  Reliability  of  Raw  Task  Factors  Across  Duty  Areas 


3* 


Duty  areas  were  prioritized  by  rank  ordering  task  factors  by  their  mean  ratings 
(see  Table  12).  The  scores  of  each  task  factor  resulted  in  a  different  rank  order  for  the  10 
duty  areas.  Task  difficulty  ratings  ranged  from  M  Duty  B  =  5-51  to  M  Duty  F  =  4.37.  The 
highest  TLD  ranking  is  Duty  J  (M  Duty  J  =  6.65)  and  the  lowest  is  Duty  F  (M  Duty  F 
=4.66).  TPD  ratings  ranged  from  M  Duty  J  =  6.45  to  M  Duty  h  =  4-13.  Several  standard 
deviations  are  also  fairly  large,  such  as  1.05  and  1.11  for  TD  and  TPD  Duty  B. 

A  general  comparison  of  tasks  rated  high  for  each  duty  area  is  listed  in  Table  13. 
As  shown,  several  tasks  were  considered  highly  difficult  (M+1SD)  and  are  common  to  the 
three  rating  schemes. 


Task  Factors  and  Criterion  Measures 

Hypothesis  III:  Testing  for  Relationships  Between  Task  Factor  Ratings  and  Percent  Time 
Spent  Criteria 


A  summary  of  the  correlations  and  regression  values  for  task  factor  ratings  and 
percent  time  spent  ratings  are  listed  in  Table  14.  Moderate  correlations  suggest  a 
relationship  of  more  time  spent  performing  the  less  difficult  tasks  and  less  time  spent 
performing  the  tasks  rated  as  highly  difficult  for  all  three  task  factor  surveys.  Using 
Fisher's  z-transformation  (zcv  =  1.96,£<  .05),  no  differences  were  found  between  the 
three  task  factor  rating  correlations  and  the  total  percent  time  spent  ratings,  accepting  the 
null  hypotheses.  The  trend  of  no  differences  was  noted  in  all  percent  time  spent  criterion 
groups  across  all  factors.  Further,  higher  order  regression  analysis  was  also  conducted 
using  the  CODAP  Curves  program.  Higher  order  polynomials  did  not  significantly 
contribute  to  regression  results. 

Analyses  within  criterion  groups  found  lower  paygrades,  less  experienced  skill 
levels,  less  senior  TAFMS,  and  TICF  personnel  had  significantly  higher  correlations  than 
their  senior  more  experienced  counterparts.  The  junior,  less  experienced  personnel 
spend  more  time  performing  the  less  difficult  tasks.  Two  exceptions  included  TLD  and 
TPD  TICF  correlations  and  TPD  TAFMS  correlations.  The  top  25  tasks  in  which  the 
senior  criterion  groups  spend  the  most  amount  of  time  performing  and  the  corresponding 
rank  order  of  task  factor  scores  are  shown  in  Appendix  G. 

Hypothesis  IV:  Testing  for  Relationships  Between  Task  Factor  Ratings  and  Percent 
Members  Performing  Criteria 

Using  Fisher's  z-transformation  test  for  independent  correlation  comparisons  at 
.05  level  of  significance,  the  null  hypothesis  could  not  be  rejected.  Correlations  between 
the  different  task  factors  for  the  percent  members  performing  criterion  were  not 
significantly  different.  Higher  order  regression  analysis,  also  using  the  CODAP  Curves 
program,  did  not  significantly  improve  predictions,  R2 .  No  differences  were  noted  in 
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relationship  with  experience  or  seniority  factors  either.  Correlation  details  and  values 
are  summarized  in  Table  15.  The  25  tasks  performed  by  the  highest  percent  of  criterion 
group  members  are  shown  in  Appendix  H. 


Table  13.  Task  Factor  Tasks  >  Raw  Mean  +1  SD  in  Each  Duty  Area 


DUTY 

TD 

TLD 

TPD 

DUTY 

TD 

TLD 

TPD 

A 

A37 

A15 

A15 

C 

C131 

C131 

C131 

A39 

A  8 

A  8 

C  98 

cm 

Cl  32 

A15 

A16 

A16 

Cl  35 

Clio 

Cl  33 

A38 

A  9 

A39 

Clio 

C  98 

Cl  35 

A  8 

A37 

A29 

C112 

Cl  35 

cm 

A  9 

A39 

A18 

Cl  07 

Cl  30 

C112 

A29 

Cl  30 

A20 

D 

D154 

D154 

D154 

B 

B94 

B80 

B80 

D143 

D176 

D176 

• 

B59 

B81 

B94 

D158 

D161 

D165 

B62 

B55 

B91 

D176 

D143 

D161 

B87 

B62 

B81 

D160 

D158 

D158 

B84 

B59 

B62 

D165 

D165 

D143 

B80 

B61 

B64 

D162 

D162 

B85 

B94 

D161 

B81 

E 

El  94 

El  94 

El  94 

B93 

El  86 

El  98 

El  98 

B74 

El  98 

El  88 

El  86 

B91 

E200 

E199 

El  88 

*  Bold  Tasks  =  Tasks  not  rated  as  "high  difficulty"  by  TD,  TLD,  and  TPD  raters 


(Non-Core  Tasks) 


39 


Table  13.  continued 


DUTY 

TD 

TLD 

TPD 

DUTY 

TD 

TLD 

TPD 

F 

F211 

F211 

F211 

G 

G398 

G362 

G358 

F251 

F245 

F247 

G401 

G358 

G396 

F273 

F247 

F265 

G400 

G361 

G370 

F247 

F265 

F314 

G366 

G373 

G395 

F245 

F317 

F317 

G362 

G351 

G400 

F314 

F261 

F318 

G351 

G366 

G398 

F318 

F314 

F275 

G361 

G396 

G351 

F261 

F251 

F261 

G373 

G398 

G355 

F317 

F316 

F251 

G393 

G400 

G362 

F272 

F315 

F319 

G395 

F319 

F249 

F249 

H 

H418 

H418 

H418 

F244 

F318 

F316 

H412 

H430 

H430 

F271 

F275 

F315 

H430 

H424 

H424 

F240 

F307 

F212 

H413 

H412 

H412 

F307 

F273 

F273 

H424 

H413 

H413 

F265 

F244 

F248 

H419 

H419 

H419 

F323 

F222 

F272 

1 

1471 

1471 

1471 

F249 

F323 

F240 

1446 

1446 

1472 

F321 

F319 

F309 

1472 

1445 

1446 

F264 

F264 

F271 

1445 

1472 

1445 

F248 

F219 

F307 

1461 

1474 

1469 

F219 

F217 

F219 

1473 

F309 

F248 

J 

J482 

J500 

J497 

J498 

J497 

J498 

J494 

J496 

J488 

J499 

J495 

J495 

J499 

J500 

J494 

J496 

Table  14.  Hypothesis  III:  Task  Factor  Correlation  with  Percent  Time  Spent  and 
Computed  R2  Values 


PTS 

r 

R2 

CRITERION 

TD 

TLD 

TPD 

TD 

TLD 

TPD 

TOTAL 

-.5852 

-.5473 

-.6165 

.3425 

.2996 

.3801 

PAYGRADE 

E-1/2/3 

-.5989 

-.5398 

-.6047 

.3587 

.2913 

.3656 

E-4/5/6 

-.5889 

-.5442 

-.6181 

.3468 

.2961 

.3820 

E-7/8/9 

-.1870 

-.2767 

-.2779 

.0350 

.0765 

.0772 

SKILL  LEVEL 

27230 

-.6063 

-.5424 

-.6079 

.3676 

.2942 

.3695 

27250 

-.5846 

-.5355 

-.6126 

.3417 

.2868 

.3753 

27270 

-.4822 

-.5032 

-.5533 

.2325 

.2532 

.3061 

28290 

-.0006 

-.0995 

-.0465 

.0000 

.0099 

.0022 

27200 

+.0597 

-.0200 

+.0088 

.0036 

.0004 

.0001 

TAFMS  (months) 

1-24 

-.5966 

-.5344 

-.5966 

.3559 

.2856 

.3560 

1-48 

-.6014 

-.5435 

-.6127 

.3617 

.2954 

.3754 

49-96 

-.5975 

-.5384 

-.6170 

.3571 

.2898 

.3806 

97+ 

-.5134 

-.5148 

-.5708 

.2636 

.2650 

.3259 

97-144 

-.5644 

-.5335 

-.6034 

.3185 

.2846 

.3641 

145-192 

-.5041 

-.5019 

-.5598 

.2541 

.2519 

.3134 

193-240 

-.4326 

-.4616 

-.4976 

.1871 

.2131 

.2476 

241  + 

-.0412 

-.1469 

-.1272 

.0017 

.0216 

.0162 

TICF  (months) 

1-24 

-.6024 

-.5410 

-.6029 

.3629 

.2927 

.3635 

1-48 

-.6044 

-.5456 

-.6146 

.3653 

.2977 

.3778 

49-96 

-.5745 

-.5293 

-.6116 

.3300 

.2801 

.3741 

97+ 

-.4808 

-.4940 

-.5449 

.2312 

.2969 
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Table  15.  Hypothesis  IV:  Task  Factor  Correlation  with  Percent  Members  Performing  and 
Computed  R2  Values 


PMP 

r 

R2 

CRITERION 

TD 

TLD 

TPD 

TD 

TLD 

TPD 

TOTAL  PMP 

.5961 

.5705 

.6396 

.3553 

.3255 

.4090 

PAYGRADE 

E-1/2/3 

-.6015 

-.5558 

-.6189 

.3619 

.3090 

.3830 

E-4/5/6 

-.5914 

-.5613 

-.6342 

.3497 

.3150 

.4022 

E-7/8/9 

-.4935 

-.5439 

-.5801 

.2436 

.2958 

.3365 

SKILL  LEVEL 

27230 

-.6078 

-.5579 

-.6247 

.3695 

.3112 

.3903 

27250 

-.5863 

-.5511 

-.6265 

.3438 

.3037 

.3925 

27270 

-.5659 

-.5770 

-.6337 

.3203 

.3329 

.4016 

28290 

-.3160 

-.4083 

-.3769 

.0999 

.1667 

.1420 

27200 

-.2712 

-.3431 

-.3655 

.0735 

.1177 

.1336 

TAFMS  (months) 

1-24 

-.6040 

-.5531 

-.6148 

.3648 

.3059 

.3780 

1-48 

-.6005 

-.5551 

-.6226 

.3606 

.3081 

.3877 

49-96 

-.5942 

-.5509 

-.6288 

.3531 

.3035 

.3954 

97+ 

-.5702 

-.5713 

-.6331 

.3252 

.3263 

.4009 

97-144 

-.5705 

-.5526 

-.6235 

.3255 

.3053 

.3887 

145-192 

-.5682 

-.5665 

-.6285 

.3229 

.3209 

.3950 

193-240 

-.5555 

-.5760 

-.6290 

.3086 

.3318 

.3957 

241  + 

-.4225 

-.4949 

-.5046 

.1785 

.2449 

.2546 

TICF  (months) 

1-24 

-.6045 

-.5574 

-.6201 

.3654 

.3107 

.3845 

1-48 

-.6012 

-.5565 

-.6250 

.3615 

.3097 

.3906 

49-96 

-.5817 

-.5469 

-.6274 

.3384 

.2991 

.3936 

97+ 

-.5720 

-.6306 

.3170 

.3272 

.3977 
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Hypothesis  V:  Testing  for  Relationships  Between  Task  Factor  Ratings  and 
Training  Emphasis  Criteria 

The  null  hypotheses  concerning  the  strength  of  relationships  between  difficulty 
task  factors  and  tasks  receiving  high  training  emphasis  which  are  also  performed  by  at 
least  30  percent  of  first  job  personnel  could  not  be  rejected.  As  shown  in  Table  16,  and 
supported  by  a  student's  t-test  (t^  =  1 .684,  df  =  44,  £  <  .05)  the  correlations  were  not 
significantly  different  from  0.  Appendix  I  lists  the  TE  tasks  receiving  the  highest  ratings 
and  shows  those  performed  by  at  least  50  percent  of  members  in  their  first  job  (1-24 
months  TAFMS).  The  rank  order  of  the  tasks  are  also  included. 

Table  16.  Hypothesis  V:  Task  Factor  Correlation  with  Training  Emphasis  of  Tasks 
Performed  by  First  Job  Personnel  and  Computed  R2  Values 


CRITERION 

TE 

r 

TD  TLD  TPD 

R2 

TD  TLD  TPD 

>30  PMP  1st  Job 

+.0941  +.1303  +.0087 

.0089  .0170  .0001 
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DISCUSSION 


Efforts  to  establish  a  new  task  difficulty  collection  procedure  which  would  improve 
the  reliability  of  data  and  provide  more  valid  measures  of  difficulty  were  partially 
successful.  Consistent  with  previous  research  which  supports  the  significance  of 
collecting  data  which  emphasize  the  learning  aspect  of  difficulty,  the  relationship  among 
task  difficulty,  task  learning  difficulty  and  task  performance  difficulty  was  examined. 
Further  comparisons  were  made  to  other  task  data,  such  as  time  spent,  seniority  and 
experience  ratings,  and  training  emphasis  factors,  to  evaluate  convergent  validity  of  the 
three  measures  of  task  difficulty.  A  discussion  of  these  results  is  presented  in  the 
following  sequence:  brief  comments  on  the  survey  response  rate,  discussions  of  intra- 
and  inter-rater  reliability,  the  relationship  between  task  difficulty  factors,  and  finally 
remarks  about  the  relationship  of  task  difficulty  factors  and  the  criterion  measures. 

Survey  Response 

The  60  percent  return  rate  projected  by  USAFOMS  historical  records  was  far 
exceeded  with  74  percent  of  the  mailed  job  inventories  and  over  69  percent  of  the  task 
factor  surveys  being  usable.  According  to  USAFOMS  procedures  for  data  Collection, 
both  major  command  (MAJCOM)  and  skill  level  distributions  must  be  represented  in  the 
survey  sample  before  administration  closure.  Achieving  a  mirror  distribution  of  these  two 
groups  allows  conclusions  to  be  generalized  to  the  entire  career  field  population. 
Unfortunately,  during  the  survey  period,  the  Air  Force  was  involved  with  restructuring  its 
major  commands.  As  a  result,  accurate  MAJCOM  distribution  data  were  not  available. 
However,  as  illustrated  in  Table  2,  the  survey  sample  is  representative  across  skill  levels. 
The  accuracy  of  the  skill  level  distribution  combined  with  the  high  survey  return  rate 
support  the  creditability  and  the  generalizability  of  the  results. 

Similar  task  factor  response  rates  were  also  necessary  to  increase  the  validity  of 
the  results.  As  shown  in  Table  3,  approximately  equal  numbers  of  respondents  across  all 
experience  and  seniority  groups  rated  each  task  factor  booklet.  As  a  whole,  task  factor 
respondents  were  AFSC  272X0,  7-skill  level,  technical  or  master  sergeants  (E-6  through 
E-8),  with  over  97  months  in  the  career  field  and  145-240  months  of  active  federal  military 
service. 

Task  Factor  Reliability 
Intrarater  Reliability 

The  first  task  factor  reliability  objective  was  to  ensure  respondents  were  using  the 
same  logic  when  rating  all  tasks  within  their  task  factor  booklet.  As  expected,  correlations 
between  all  duplicate  tasks  were  very  high.  This  indicates  that  within  rating  scale  types, 
the  same  thought  process  was  generally  used  in  rating  all  the  tasks;  i.e.,  the  ratings  are 
reliable  regardless  of  the  type  of  difficulty  rating  instruction.  Task  Learning  Difficulty 
(TLD)  and  Task  Difficulty  (TD)  raters  demonstrated  stronger  test-retest  reliability  when 
compared  to  Task  Performance  Difficulty  (TPD).  The  lower  correlations  between  TPD 
task  ratings  imply  the  various  aspects  of  performance  difficulty  may  somewhat  impact  the 
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consistency  of  the  thought  process  when  rating  several  tasks.  TPD  raters  may  have 
more  difficulty  identifying  the  reasons  tasks  are  difficult  to  perform  and  consistently 
applying  that  rating  rationale  throughout  the  TPD  inventory  compared  to  the  TD  and  TLD 
inventories. 

Several  researchers,  such  as  Christal  (1974)  arid  Ruck,  Thompson,  and 
Thompson  (1978),  have  determined  that  a  minimum  of  20  to  40  raters  is  needed  to  obtain 
reliable  task  factor  information.  More  raters  (30  +)  increases  the  likelihood  that  the  data 
will  be  usable.  While  more  than  20  respondents  rated  the  24  duplicate  tasks,  only  five 
tasks  were  rated  by  greater  than  30  TD  raters.  This  was  significantly  less  than  the  21 
tasks  rated  by  30  TLD  respondents.  This  difference  did  not  impact  the  results  of  this 
study,  but  in  context  of  future  surveys,  the  greater  number  of  respondents  rating  tasks  as 
demonstrated  by  the  TLD  inventory,  may  reduce  the  probability  of  collecting  difficulty  data 
which  can  not  be  used  because  an  insufficient  number  of  raters  rated  the  tasks. 

Interrater  Reliability 


The  second  reliability  study  objective  was  to  determine  the  interrater  reliability  of 
the  task  factor  data  after  removing  divergent  raters.  Interrater  reliability  or  R^  was  "high" 
in  all  three  samples  and  the  resulting  reliability  of  any  one  rating  scheme  was  not  more 
reliable  than  the  others.  However,  in  achieving  these  high  interrater  reliabilities,  six 
divergent  raters  were  removed  from  the  TD  sample.  This  was  significantly  more  than  the 
single  rater  removed  from  the  TLD  sample  and  may  indicate  that  the  TD  instructions 
produce  more  divergent  raters. 

CODAP  GRPREL  also  has  the  capability  to  determine  the  minimum  number  of 
raters  that  would  have  been  required  to  establish  an  R^  of  .90.  In  this  study,  only  26  TD, 
23  TLD,  and  19  TPD  raters  were  necessary.  Research,  as  previously  mentioned,  claims 
that  under  normal  situations,  survey  administrators  and  analysts  should  budget  for  a 
minimum  of  20  to  40  raters  to  ensure  rater  reliability.  The  three  surveys  averaged  ratings 
of  over  30  raters  per  task  and  easily  met  this  requirement.  But  on  a  task  level  review, 
approximately  1 0  percent  of  the  tasks  were  rated  by  less  than  20  raters  in  the  TD  and 
TPD  surveys.  TLD  respondents  only  fell  below  20  raters  on  less  than  5  percent  of  the 
tasks.  As  eluded  to  earlier,  respondents  are  not  always  able  to  rate  every  task  due  to 
lack  of  task  knowledge  or  perhaps  lack  of  comfort  with  the  rating  scheme.  This  tendency 
for  TLD  sample  to  include  more  raters  per  task  as  well  as  less  divergent  raters  per  survey 
minimizes  the  chance  that  task  factor  data  will  be  rejected  due  to  an  insufficient  number 
of  usable  raters. 

The  TLD  survey  appears  to  be  a  more  reliable  method  for  collecting  difficulty 
data.  Ratings  are  more  consistent,  data  have  high  interrater  reliability,  fewer  divergent 
raters,  and  survey  tasks  have  a  tendency  to  be  rated  by  a  higher  percentage  of  the  raters. 
These  results  suggest  that  guidance  and  design  of  the  TLD  booklet  results  in  the 
collection  of  more  reliable  data. 


Relationships  Between  Task  Factor  Ratings 


As  expected,  correlation  analysis  identified  a  relationship  between  task  factor 
booklets.  Unexpectedly,  the  correlations  were  "very  high"  with  the  smallest  correlation 
between  TD  and  TPD  surveys  and  the  largest  between  TLD  and  TPD  surveys. 

Nearly  80  percent  of  the  TD  variance  could  be  explained  by  the  TLD  survey  and 
approximately  77  percent  could  be  explained  by  the  TPD  ratings.  This  was  surprising 
since  research  and  expert  opinions  suggested  that  the  less  specific  "task  difficulty"  would 
consist  of  aspects  of  both  learning  and  performance  difficulty,  with  respondents  inclining 
towards  rating  the  performance  aspect  of  difficulty.  The  lack  of  a  significant  difference 
between  TD  and  TLD  versus  TD  and  TPD  correlations  did  not  confirm  this  line  of  thinking, 
suggesting  maybe  the  brief  reference  to  task  learning  difficulty  which  is  included  in  TD 
survey  instructions  may  have  had  some  impact  on  the  ratings. 

Even  more  surprising  is  the  significantly  greater  correlation  between  TLD  and 
TPD  then  TD  and  TPD.  Research  (Burtch,  Lipscomb,  &  Wissman,  1982;  Christal,  1970, 
1974;  Lecznar,  1971;  Madden,  1962;  Mead,  1970a,  1970b;  Weeks,  1981)  has  indicated 
the  need  to  clarify  and  collect  the  learning  aspect  of  difficulty  because  of  both  increased 
reliability  and  understanding  and  utility  of  the  data.  According  to  the  correlational  results, 
respondents  rate  learning  and  performance  aspects  of  a  task  basically  the  same.  Over 
90  percent  of  the  variance  can  be  explained  between  TLD  and  TPD  ratings.  Since  10 
percent  is  still  unaccounted  for,  another  approach  was  needed  to  assess  the  surveys. 

The  following  sections  examine  tasks  which  were  rated  differently  under  the  three  survey 
methods. 

Task  Factor  Distributions 

As  an  aid  to  the  discussion,  the  practical  applications  of  the  survey  results  should 
be  considered.  In  determining  training  and  testing  needs,  USAFOMS  advises  career  field 
subject  matter  experts  to  seriously  evaluate  tasks  with  ratings  1  or  2  standard  deviations 
above  the  mean,  i.e.,  high  difficulty  tasks,  when  making  personnel  management 
decisions.  While  tasks  with  mean  ratings  or  lower  should  also  be  reviewed  and 
discussed,  tasks  with  ratings  greater  than  1  standard  deviation  below  the  mean  can 
normally  be  disregarded.  As  shown  in  Table  7,  79  TD,  76  TLD,  and  78  TPD  tasks  meet 
the  high  difficulty  requirements. 

Task  Data  Comparisons  Across  Survey  Booklet 

Of  the  tasks  which  were  rated  "high"  on  the  different  difficulty  scales,  43  were 
common  to  all  surveys.  Of  the  remaining  tasks,  10  tasks  were  common  to  only  TD  and 
TLD,  8  common  to  just  TD  and  TPD,  and  19  common  to  TLD  and  TPD.  This  left  19 
unique  TD  tasks,  3  unique  TLD  tasks,  and  9  unique  TPD  tasks.  Following  are  lists  of  the 
unique  tasks: 


TD:  A038  Prepare  or  submit  recommendations  for  improving  or  standardizing  ATC 

procedures 
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B084  Supervise  Air  T raffle  Control  Automated  Systems  Programming 
Specialists  (AFSC  D272X0) 

B085  Supervise  Air  Traffic  Control  Managers  (CEM  27200) 

B086  Supervise  Air  Traffic  Control  Operators  (AFSC  27250) 

B088  Supervise  Air  Traffic  Control  Technicians  (AFSC  27270) 

C097  Administer  facility  rating  examinations  or  position  certifications 

Cl  01  Conduct  staff  assistance  or  site  visits 

Cl  02  Evaluate  activity  reports 

Cl  03  Evaluate  aircraft  save  requests 

Cl  06  Evaluate  ATC  complaints 

Cl  08  Evaluate  ATC  recommendations 

Cl  1 8  Evaluate  mission  impact  resulting  from  ATCALS  deficiencies 
D160  Develop  resident  course  curriculum  materials 
E200  Prepare  Ols 

H418  Authorize  simultaneous  opposite  direction  operations 

1442  Compute  PAR  voltages 

1448  Erect  or  tear  down  cantonment  facilities 

1467  Plan  aircraft  loading  or  unloading 

1473  Review  operation  orders  or  plans 


TLD:  D161  Direct  or  implement  OJT  programs 
1474  Set  up  disaster  control  facilities 
G362  Control  timed  approaches 


TPD:  A007 

A018 

A029 

B066 

B091 

F265 

G358 

1468 

J491 


Coordinate  air  traffic  control  (ATC)  procedures  for  disaster  control 

exercises  with  other  agencies 

Establish  facility  mission  impact  guidelines 

Plan  for  ATC  analysis  visits 

Implement  cost-reduction  programs 

Supervise  civilian  personnel 

Erect  or  tear  down  TRN-41  mobile  tactical  air  navigation  system 
(TACANs) 

Control  air  traffic  using  oceanic  procedures 
Position  mobile  ATC  equipment  or  support  equipment 
Maintain  data  bases  for  automated  ATC  systems 


Several  of  the  unique  high  TD  and  TPD  tasks  were  characteristic  of  certain 
activities.  Specifically,  1 3  of  the  1 9  tasks  or  68  percent  of  the  unique  TD  tasks  are 
supervisory  or  managerial  activities  from  Duties  A  through  D:  (A:  1,  B:  4,  C:  7:,  D  1). 
Similarly,  five  of  the  10,  or  50  percent,  of  the  unique  TPD  tasks  are  also  supervisor  and 
managerial  duties. 


Discussion  with  a  subject  matter  expert  noted  tasks  rated  uniquely  high  by  both 
TLD  and  TPD  raters,  especially  those  in  Duty  J  were  both  difficult  to  learn  and  to  perform 
Whereas  the  tasks  unique  to  TPD,  F265  and  1468  were  relatively  easier  to  learn  but 
physically  difficult  or  cumbersome  to  perform.  These  tasks  in  particular  appear  to  be  a 
variation  of  the  "bolt  in  the  plane"  scenario  in  which  the  airman  is  taught  that  a  certain 
task  simply  involves  switching  out  black  boxes  in  the  plane  but  because  the  boxes  were 
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bolted  in  the  plane  at  an  angle  very  difficult  to  reach,  the  task  is  considered  difficult  to 
perform.  Similarly,  the  unique  TLD  tasks  (G362, 1474)  were  considered  slightly  more 
involved  or  harder  to  learn,  but  once  understood  fairly  easy  to  accomplish. 

The  overlap  between  TLD  and  TPD  high  difficulty  tasks  is  not  as  easily 
explained  but  may  be  a  result  of  raters  rating  all  and  any  aspects  of  difficulty.  Of  practical 
concern  is  the  impact  of  applying  these  ratings  in  policy  decisions.  Specifically,  several 
tasks  which  were  considered  difficult  to  leam,  based  on  TLD  ratings,  were  not  captured 
under  the  current  TD  survey  guidance  as  "highly"  difficult.  Several  TD  tasks,  on  the  other 
hand,  would  have  been  recommended  for  inclusion  in  training  and  testing  documents  but 
are  not  difficult  to  learn  based  on  the  TLD  ratings.  This  discrepancy  is  confirmed  when 
tasks  are  examined  by  duties. 

Task  Data  Comparisons  Within  Duty  Areas 

The  survey  data  were  divided  into  duty  areas  to  examine  interrater  reliability  by 
duties.  In  general,  reliability  was  lower  and  a  greater  number  of  divergent  raters  removed 
from  each  duty  sample.  Importantly,  the  TLD  raters  maintained  high  reliability  for  five 
duties  versus  only  four  for  TPD  and  two  for  TD.  Further,  while  two  duties  had  negligible 
reliability  for  TD  and  TPD,  only  one  duty  demonstrated  poor  reliability  for  TLD.  At  this 
more  macro  level  of  analysis,  TLD  again  appears  to  be  the  more  reliable  of  the  rating 
methods. 


Duty  areas  were  placed  in  rank  order  of  average  task  factor  ratings.  The  result 
was  a  different  rank  order  of  duty  areas  for  each  difficulty  factor.  Whereas  TD  ratings 
suggest  Duty  B,  Directing  and  Implementing,  and  Duty  C,  Inspecting  and  Evaluating,  are 
the  most  difficult.  TLD  and  TPD  ratings  indicate  Duty  J,  Performing  Air  Traffic  Control 
Data  Programming  Functions,  and  Duty  I,  Performing  Mobile  Operations,  and  Duty  C  are 
the  most  difficult.  Duties  B,  C,  I,  and  J  were  ranked  in  the  top  four  duties  by  both  TLD 
and  TPD  ratings.  The  differences  with  low  difficulty  levels  are  minimal  with  the  three 
survey  methods  rank  ordering  Duties  G,  Performing  Radar  Functions,  Duty  H,  Perform 
Control  Tower  Functions,  and  Duty  F,  Performing  General  Air  Traffic  Control  Functions, 
as  either  8,  9,  or  10. 


TD  TLD  TPD 
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The  standard  deviations  per  duty  area  range  from  .18  (TPD  Duty  J)  to  1.11  (TD 
Duty  B).  Most,  however,  are  large  enough  to  impact  the  rank  order  of  tasks  across 
duties.  For  example,  a  highly  difficult  TPD  task  in  Duty  B  would  rank  higher  than  a  highly 
difficult  TPD  task  in  Duty  C.  This  was  seen  in  the  representation  of  several  duties  in  the 
tasks  considered  highly  difficult  in  Tables  8,  9, 10. 

Comparison  of  duty  area  tasks  was  completed  using  a  technique  similar  to  the 
method  used  to  compare  tasks  rated  as  high  (M+1SD)  for  the  three  survey  booklets. 
Since  Duties  B,  C,  J,  and  I  were  rated  as  the  four  most  difficult  duty  areas,  they  were 
examined  first.  As  shown  in  Table  13,  a  total  of  17  TD  task,  14  TLD  tasks,  and  14  TPD 
tasks  were  not  rated  as  highly  difficult  by  all  three  rating  methods.  Nine  TD  tasks,  three 
TLD  tasks,  and  five  TPD  tasks  received  an  average  rating  of  at  least  1  standard  deviation 
above  the  mean  in  the  four  duty  areas  by  one  rating  method.  Several  tasks  were 
common  to  only  two  rating  methods.  Specifically,  six  tasks  were  considered  difficult  by 
both  TLD  and  TPD  raters,  five  by  TD  and  TLD  raters,  and  three  by  TD  and  TPD  raters. 
Review  of  these  tasks  or  tasks  rated  as  highly  difficult  by  only  one  or  two  rating  methods 
found  that  most  were  identified  earlier  in  the  Task  Data  Comparisons  Across  Survey 
Booklet  section. 


The  unique  tasks  not  identified  in  the  survey  level  analysis  include: 


TD:  B087 
B093 
Cl  07 
1461 
J482 


Supervise  Air  traffic  Control  Superintendents  (AFSC  27290) 
Supervise  Combat  Control  Operators  (AFSC  27350) 
Evaluate  ATC  problem  areas 
Operate  site  survey  equipment 
Build  operational  programs 


TLD:B055 

B061 


Direct  conventional  air  route  traffic  control  activities 
Direct  radar  air  route  traffic  control  activities 


TPD:B064 

Cl  32 

Cl  33 

1469 

J488 


Direct  tower  instrument  flight  rules  (IFR)  approach  control 
activities 

Prepare  enlisted  performance  reports  (EPRs)  or  letters  of  evaluation 
(LOEs) 

Prepare  recommendations  for  awards  or  decorations 
Prepare  mission  limiting  factors 

Configure  EARTS  hardware  systems  for  operational  needs 


Since  the  managerial  and  supervisory  type  tasks  are  difficult  to  assess  in  reference  to 
difficulty,  only  task  1 469  and  J488  were  discussed  with  a  SME.  Both  tasks  were 
perceived  to  be  more  difficult  to  perform  than  to  learn. 


Finally,  the  least  difficult  duties  were  examined  for  similar  trends.  Relatively, 
Duty  Areas  F,  G,  and  H  were  rank  ordered  as  the  lowest  in  difficulty,  however  several 
tasks  within  each  duty  were  rated  as  highly  difficult.  Most  tasks  in  all  three  duties  were 
considered  difficult  by  the  three  rating  schemes.  Of  particular  interest  was  the  unique 
difficult  tasks  or  tasks  rated  as  difficult  by  only  one  method: 
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TD:  F321  Provide  wake  turbulence  separations 

G393  Perform  radar  equipment  turnaround  procedures 
G401  Provide  radar  service  for  arrivals 

TLD:  F217  Approve  or  coordinate  aerial  refueling  operations 

F222  Approve  or  coordinate  special  visual  flight  rules  (SVER)  operations 

TPD:F212  Apply  visual  separations 

G355  Conduct  military  training  route  (MTR)  operations 

G370  Implement  air  defense  identification  zone  (ADIZ)  procedures 

Again,  the  differences  could  be  explained  by  the  nature  of  the  task  and  its  rating  scheme 
for  which  it  was  rated  difficult. 

In  general,  results  of  duty  level  task  comparisons  supported  the  findings  of  the 
survey  level  task  comparisons.  TLD  data  had  higher  interrater  reliability  across  duties 
and  differences  in  unique  tasks  within  duties  were  identified  as  related  to  the  differences 
in  rating  schemes.  Based  on  duty  means  and  standard  deviations,  tasks  rated  high  on 
learning  difficulty  were  not  captured  using  the  current  TD  survey  procedure.  Since  duty 
areas  are  not  normally  used  by  decision  makers  to  identify  training  or  testing  needs,  the 
practical  implication  of  these  differences  is  minimal.  However,  the  results  confirm  the 
need  to  review  historic  data  to  determine  if  learning  difficulty  tasks  have  been  omitted 
during  the  decision  process  or  if  high  performance  difficulty  tasks  which  are  captured 
under  the  current  procedures  were  included. 

Finally,  the  practical  differences  identified  in  this  Air  Force  Specialty  suggests 
further  research  is  needed  to  investigate  the  impact  of  different  difficulty  instructions  and 
survey  format  on  task  ratings  on  other  specialties.  Furthermore,  the  historical  impact  of 
implementing  decisions  based  on  tasks  considered  highly  difficult  under  the  current  task 
difficulty  collection  procedures  which  includes  tasks  which  were  both  difficult  to  learn  and 
difficult  to  perform  needs  to  be  investigated. 

Relationship  Between  Task  Factors  and  Criterion  Measures 

The  effort  to  use  criterion  measures  to  determine  the  rating  logic  of  respondents 
and  therefore  the  utility  of  the  current  task  difficulty  format  as  well  as  the  proposed  task 
learning  difficulty  survey  format  was  not  particularly  successful.  The  relationship  of  the 
difficulty  ratings  with  the  three  key  criterion  measures:  percent  time  spent,  senior  time 
factors,  and  training  emphasis  ratings  were  moderate,  were  generally  in  the  hypothesized 
directions,  but  were  not  significantly  different  between  the  three  survey  approaches. 

Relationship  with  Percent  Time  Spent  Criteria 

Although  past  research  was  limited,  Madden  (1961)  and  McCauley,  O'Leary,  and 
Rheinstein  (1991)  found  positive  correlations  between  task  difficulty  and  percent  time 
spent  performing.  The  current  study  analyses  revealed  that  the  more  time  individuals 


spent  on  a  task,  the  less  difficult  the  task  was  rated  under  all  three  survey  formats.  A 
probable  explanation  of  this  phenomenon  might  be  that  in  this  career  field,  individuals 
spend  a  lot  of  time  performing  many  tasks  which  with  experience  become  perceived  as 
simple  or  routine  tasks.  Examination  of  the  raw  data  (see  Appendix  H)  indicates  that,  in 
general,  the  high  difficult  tasks  (M  +  1SD)  or  tasks  ranked  higher  than  79  for  TD,  76  for 
TLD,  and  78  for  TPD  were  not  performed  by  a  high  percentage  of  personnel.  Of  the  top 
25  tasks  performed  by  the  different  paygrade  groups,  only  two  tasks  received  high 
difficulty  ratings  from  each  task  factor  survey. 

Relationship  with  Percent  Members  Performing  Criteria 

The  only  criterion  relationship  in  the  study  which  somewhat  supports  the  literature 
(Dittmar,  Driskill,  &  Weismuller,  1987;  Ruck,  Thompson,  &  Stacy,  1987)  was  aspects  of 
time.  As  demonstrated  by  the  negative  correlation  between  time  spent  performing  and 
difficulty  ratings  and  the  significant  difference  in  correlation  of  junior  and  senior  time 
factors,  the  assumption  that  less  experienced  or  junior  personnel,  based  on  paygrade, 
skill  level,  Total  Active  Federal  Military  Service  (TAFMS),  and  Time  in  Career  Field 
(TICF),  spend  more  time  on  routine  less  difficult  tasks  holds  true. 

Apparently,  while  junior  personnel  spend  less  time  on  the  more  difficult  tasks  and 
more  time  on  the  less  difficult  tasks,  senior  personnel  are  spending  a  varying  amount  of 
time  on  tasks  with  varying  degrees  of  difficulty.  So  while  the  basic  hypothesis  involving 
the  difference  in  the  relationship  of  task  factor  ratings  to  these  time  factors  is  still  not  fully 
supported,  the  nature  of  task  difficulty  rating  was  somewhat  clarified. 

Correlations  with  percent  members  performing  data  for  the  time  groups  did  not 
add  to  an  understanding  of  the  task  difficulty  measures.  In  fact,  the  lack  of  significant 
differences  between  percent  of  junior  and  senior  personnel  performing  tasks  indicates 
that  with  the  moderate  correlation,  fewer  incumbents  perform  the  high  difficulty  tasks 
regardless  of  experience  or  time  in  the  military  or  career  field.  Since  it  was  foreseeable 
that  a  curvilinear  relationship  might  exist,  a  higher  order  regression  analysis  was 
conducted  and  R2  computed,  but  there  were  no  curvilinear  relationships  present. 

Returning  to  the  task-level  comparison  across  duty  areas,  duties  such  as  I  and  J 
as  well  as  the  supervisory  and  managerial  functions  were  ranked  as  most  difficult  and  are 
associated  with  the  more  experienced  and  senior  personnel.  On  the  other  hand,  general 
duties  such  as  F,  G,  and  H  were  ranked  lowest  and  are  primarily  considered  tasks 
performed  by  junior  personnel.  While  the  relationship  was  not  significant,  the  task  view 
approach  supports  the  concept  that  more  difficult  tasks  are  performed  by  senior 
personnel.  Unfortunately,  no  generalizations  of  this  relationship  can  be  drawn  to  explain 
the  differences  in  the  difficulty  measures. 

Relationship  with  Training  Emphasis  Criteria 

The  negative  correlation  proposed  by  Ruck  et  al.  (1987)  between  training 
emphasis  ratings  of  tasks  performed  by  at  least  30  percent  of  personnel  in  their  first  jobs 
(1-24  TAFMS)  did  not  exist  for  this  specialty.  In  fact,  no  relationship,  positive  or  negative, 
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was  found.  Once  again,  examination  of  the  rank  order  data  (Appendix  K)  supports  an 
earlier  comment  proposing  that  incumbents  were  not  performing  a  sufficiently  high 
percentage  of  difficult  tasks  to  make  any  comparisons. 

The  lack  of  a  clear  relationship  between  task  difficulty  factors  and  the  criterion 
measures  and  the  strong  correlations  between  the  different  measures  of  difficulty  resulted 
in  an  inability  to  draw  conclusions  about  training  emphasis  differences  among  the  survey 
metnods.  Further  research  is  necessary  with  emphasis  on  measuring  difficult  tasks 
performed  by  a  significant  number  of  personnel  in  a  heterogeneous  career  field  to 
address  this  issue. 

Summary 

The  three  major  goals  of  this  research  were  to  increase  the  reliability  of  task 
difficulty  data,  determine  if  any  differences  exist  between  methods  of  data  collection  and 
identify  content  differences  of  the  different  task  difficulty  survey  procedures.  Results 
indicated  that  TLD  measures  were  generally  more  reliable  with  a  higher  percentage  of 
raters  rating  each  task,  a  higher  percentage  of  raters  returning  usable  survey  booklets,  a 
lower  number  of  divergent  raters,  and  rating  logic  consistency.  Differences  between  data 
collection  methods  were  small  but  significant  with  a  stronger  relationship  between  TLD 
and  TPD  than  with  TD.  The  few  differences  in  tasks  rated  high  in  difficulty  were  a  result 
of  the  type  of  difficulty  used  for  ratings.  Criterion  measures  collected  to  validate  the 
ratings  provide  somewhat  inconclusive  evidence  of  the  reasons  for  differences  among 
task  difficulty  survey  methods.  So  most  conclusions  that  the  TLD  provides  a  more 
accurate  measure  of  the  desired  task  difficulty  are  based  on  analyses  of  the  obtained 
rating  data. 

The  practical  impact  of  these  findings  suggest  that  TLD  data  collection  may  be 
more  efficient.  The  TLD  procedures  require  smaller  sample  sizes,  can  provide  higher 
reliabilities,  yield  fewer  divergent  raters,  and  result  in  a  larger  number  of  tasks  being  rated 
by  each  rater.  Tasks  rated  high  in  learning  difficulty  are  not  necessarily  captured  under 
the  current  TD  format.  Advising  training  and  aptitude  standards  setting  personnel  to  key 
on  the  M+1SD  tasks  may  result  in  poor  decisions  if  the  TD  survey  method  was  used  to 
collect  that  data.  The  lack  of  a  clear  criterion  to  test  the  validity  of  difficulty  measures 
creates  problems  in  stating  the  full  utility  of  the  data  for  establishing  personnel  policies. 

While  further  research  is  warranted,  personnel  at  USAFOMS  can  revise  the 
format  of  task  difficulty  surveys  to  include  a  new  cover  page,  instructions,  and  scale 
headings  to  emphasize  the  learning  aspect  of  task  difficulty.  These  changes  can  be 
made  with  confidence  that  future  data  will  be  more  reliable  and  that  the  validity  will  be  the 
same,  or  greater,  than  data  collected  in  the  past. 
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BACKGROUND  ITEMS 


1)  Job  Inventory  Biographical  Items 

2)  Job  Inventory  Background  Items 
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JOB  INVENTORY  BIOGRAPHICAL  ITEMS 


Last  Name,  First  Initial 
Social  Security  Number 
Phone  Number 
Sex 
Grade 

Title  of  Present  Job 

Number  of  People  You  Directly  Supervise 
Time  in  Present  Job 
Time  In  Career  Field 

Total  Active  Federal  Military  Service  (TAFMS) 

Primary  AFSC 
Duty  AFSC 
Converted  AFSC 

Organization  To  Which  Assigned 
Base  To  Which  Assigned 
Command  To  Which  Assigned 
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JOB  INVENTORY  BIOGRAPHICAL  ITEMS 

Assigned  in  or  outside  the  continental  United  States 

Eligible  to  reenlist 

Eligible  for  retirement 

Job  interest 

Job  utilize  your  talents 

Job  utilize  your  training 

Satisfied  with  sense  of  accomplishment  from  work 
Plan  to  reenlist 

How  assigned  to  present  career  ladder 

Community  College  of  the  Air  Force  status 

Work  area  where  spend  most  time 

Title  of  present  position 

Type  of  facility  of  present  assignment 

Formal  courses  completed 

Control  tower  equipment  operated  in  present  job 

Navigational  aid  remote  status  indicators  used  in  present  job 

Radar  equipment  used  in  present  job 

Radio  communication  equipment  used  in  present  job 

Forms  used  in  present  job 
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APPENDIX  B:  AFSC  272X0  TASK  DIFFICULTY  (TD)  BOOKLET 


1)  TD  Cover  Page 

2)  TD  Instructions 

3)  TD  Scale  Headings 
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INSTRUCTIONS  FOR  RATING  TASK  DIFFICULTY 


As  a  senior  technician,  you  have  been  selected  to  provide  needed  information 
pertaining  to  the  difficulty  of  tasks  performed  in  your  career  ladder.  This 
information  will  be  of  value  to  the  Air  Force  in  the  improvement  of  training, 
classification,  and  testing  programs. 

To  obtain  the  maximum  response  possible,  it  is  requested  'that  you  rate  each 
task  of  which  you  have  any  knowledge.  Rate  those  tasks  you  presently  perform 
or  supervise,  those  tasks  which  you  have  performed  at  a  prior  time  in  your 
career,  and  those  tasks  which  you  have  observed  or  supervised  being  performed 
by  others.  Most  personnel  with  your  experience  and  background  will  be  able  to 
rate  the  majority  of  the  tasks  listed  and,  in  many  cases,  to  rate  all  of  them. 
To  accomplish  this  rating,  follow  the  procedure  listed  below. 


STEP  1.  For  this  survey,  task  difficulty  is  defined  as  the  amount  of  time 
needed  to  learn  to  do  each  task  satisfactorily.  To  develop  a  frame  of 
reference  for  rating  task  difficulty,  scan  the  entire  listing  of  tasks.  Pick 
out  some  easy  tasks  and  some  difficult  tasks.  Then,  select  some  tasks  that 
fall  between  these  extremes  which  are  of  average  difficulty.  Use  those  tasks 
at  or  near  the  middle  of  the  range  as  a  reference  point  for  judging  the 
difficulty  of  all  tasks  in  the  inventory.  Use  this  frame  of  reference  for 
completing  STEP  2. 


STEP  2.  Estimate  the  task  difficulty  rating  for  each  task  compared  with  other 
tasks  in  this  inventory.  Use  the  scale  shown  here  and  at  the  top  of  each  page 
to  rate  each  task. 


1. 

Extremely  Low 

2. 

Very  Low 

3. 

Low 

4. 

Below  Average 

5. 

Average 

6. 

Above  Average 

7. 

High 

8. 

Very  High 

9. 

Extremely  High 

Begin  with  the  first  task  in  the  booklet  and  give  each  task  of  which  you  have 
knowledge  a  difficulty  rating  from  1  to.  9;  record  the  value  opposite  the  task 
statement  in  the  column  titled  "TASK  DIFFICULTY."  Try  to  rate  every  task  on 
each  page.  Remember  (from  STEP  1)  that  you  are  comparing  each  task  with  the 
other  tasks  in  the  career  field. 

STEP  3.  Add  any  tasks  you  believe  are  performed  by  members  of  your  specialty 
and  are  not  covered  by  the  booklet  on  the  last  page;  then  rate  them. 

STEP  4.  Review  the  booklet  to  see  that  you  have  rated  the  DIFFICULTY  OF  ALL 
TASKS  POSSIBLE.  Each  task  can  be  given  only  one  rating. 


TASK  DIFFICULTY  RATING 


INSTRUCTIONS 

ar!  8  d^y  and  the  iasks  whleh  H  includes.  Rate  each  task  for 
difficulty  based  on  tine  needed  to  learn  to  do  the  job. 


A.  ORGANIZING  AND  PLANNING  "  “ 

•  Assign  additional  duties  .  - - - - - - 

•  Assign  personnel  to  duty  positions  “  — ---• 

•  Assign  sponsors  for  newly  assigned  personnel  ' '  •. 

•  Coo rd Ina  t e  ai r  t  r  a  f  f 1 c  coin t to  1  ( AT Cy^procedures^fo r*”d i sast eir 
_ -gontrol  exercise s  wl t h  oth e r  agencies 

.  Coordinate  aircraft  control  priorities  with  base  flying  i 1 
authorities  •;  ■;  :r:  f1  p 

.  Coordinate  airlift  operations  "with" "other 'agenciesTitt^'a^ 
— ggggggj. ..  posts  or  a irlift  contro  1  centers  (ALCCs) 

.  Coordinate  revisions  for  specie  operations  with~base  flv ini 
authorities  ;  •  7 

.  Coordinate  revisions  for  speciai  operations^with  host  'nStion 
— ®?_.'®S®ral_Aviation _  Administration  (FAA)  agencies 

Design  nap  overlays  '  . .  ™  . .  ~  ■.  ■ 

Determine  requirements  for  space,  personnel.equipment.or 
. supplies _ 

Determine  work  p r i or i ties  ; .  — — - 

Develop  facility  reference  charts'.'  "  “  *  ~ 

Develop  organizational  charts  *  — - —  ■  ~  - 

Develop  work  methods  or  procedures . ^  '  ~ 

^ra^t  budget;  or  financial  requirements  i  ~  '  ~~ ~ 7 ’ 

Establish  dispersed  controller  programs 
:  Establish  facility  documentation  fl les  ~  :  "  7  :  :  7 

Establish  facility  miss Ion  impact' guidelines'  ~ 

Establish  local  procedures  for  “controlling  flighFcheck  _ 

•  .Aircraft  :  Sv  '  ■. .  ■  \ 

rn^a^lsh  or8anlzati°nai  policies,  operating  instructions^ 
A2i?.2 > -  °.£--?.?auding  oper ating  procedures  (SOPs ) 

Establish  performance  standards  for  subordinates 

Establish  priorities  for  restoring  equipment 

Es tab llah  p rocedures  for^onTroTlTiig  traffic  between  sectors 
or  other  facilities  ; 


1  of  24  Pages 


TASK  DIFFICULTY 


Extremely  Low 
Very  Low 
Low 

Below  Average 
Average 
Above  Average 
High 

Very  High 
Extremely  High  L- 

xxxxxxxxxxxxxxxxl 
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3)  TLD  Scale  Headings 
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INSTRUCTIONS 

FOR  RATING  TASK  LEARNING  DIFFICULTY 


As  a  senior  technician,  you  have  been  selected  to  provide  needed  information 
pertaining  to  the  difficulty  of  tasks  performed  in  your  career  ladder.  This 
information  will  be  of  value  to  the  Air  Force  in  the  improvement  of  training, 
classification,  and  testing  programs.  To  accomplish  this  rating,  follow  the 
procedure  listed  below. 

NOTE:  To  obtain  the  maximum  response  possible,  it  is  requested  that  you  rate  each 
task  of  which  you  have  any  knowledge.  Rate  those  tasks  you  presently  perform  or 
supervise,  those  tasks  which  you  have  performed  at  a  prior  time  in  your  career,  and 
those  tasks  which  you  have  observed  or  supervised  while  being  performed  by 
others.  Most  personnel  with  your  experience  and  background  will  be  able  to  rate 
the  majority  of  the  tasks  listed  and  in  many  cases  to  rate  all  of  them. 

STEP  1.  Develop  a  frame  of  reference  for  rating  task  learning  difficulty.  For  this 
survey,  task  learning  difficulty  is  defined  as  the  amount  of  t[me  needed^  to  learn  to 
do  each  task  satisfactorily..  To  develop  a  frame  of  reference,  scan  the  entire  listing 
Pick  out  some  easy  tasks  and  some  difficult  tasks.  Then,  find  some  tasks  which  fall 
between  these  extremes  that  are  of  average  difficulty.  Use  these  tasks  at  or  near 
the  middle  of  the  range  as  a  reference  point  for  judging  the  learning  difficulty  of  all 
tasks  in  the  inventory.  This  frame  of  reference  will  be  used  for  completing  STEP  2. 

STEP  2.  Estimate  the  task  learning  difficulty  rating  for  each  task  compared  with 
other  tasks  in  this  inventory.  Use  the  scale  shown  here  and  at  the  top  of  each  page 
to  rate  each  task. 


1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 
9. 


Extremely  Low 
Very  Low 
Low 

Below  Average 
Average 
Above  Average 
High 

Very  High 
Extremely  High 


Begin  with  the  first  task  in  the  booklet  and  give  each  task  of  which  you  have 
knowledge  a  learning  difficulty  rating  from  1  to  9;  record  the  value  opposite  the  task 
statement  in  the  column  titled  "TASK  LEARNING  DIFFICULTY."  Try  to  rate  every 
task  on  each  page.  Remember  (from  STEP  1)  that  you  are  comparing  each  task  with 
the  other  tasks  in  the  career  field. 

STEP  3.  The  last  page  of  the  booklet  is  available  to  add  any  tasks  you  do  now 
which  are  not  listed.  Your  constructive  suggestions  in  improving  the  job  inventory 
will  be  useful. 

STEP  4.  Review  the  booklet  to  see  that  you  have  rated  the  LEARNING  DIFFICULTY 
of  all  tasks  possible.  Each  task  can  be  given  only  one  rating. 


ii 


67 


TASK  LEARNING  DIFFICULTY  RATING 


INSTRUCTIONS 

listed  below  are  a  duly  and  Ihe  tasks  which  it  Includes.  Rate  each  task  for 
learning  difficulty  based  on  tine  needed  to  learn  to  do  the  job. 


A.  ORGANIZING  AND  PLANNING 


I  of  24  Pages 


TASK  LEARNING 
DIFFICULTY 


Extremely  Low 
Very  Low 
Low 

Below  Average 
Average 
Above  Average 
High 

Very  High 
Extremely  High 


1.  Assign  additional  duties  .  . 

2.  Assign  personnel  to  duty  positions  “  “  "  . . 

3.  Assign  sponsors  for  newly  assigned  personnel 

4.  Coordinate  air  traffic  control  ( AT C )  procedures  for  disaster 

_ _ _ gontrol  exercises  ylth_ other  agencies 

5.  Coordinate  aircraft  control  priorities  wit h~basefIyTnK 
_ authorities  ' 

6.  Coordinate  airlift  operations  with  other  agencies ,  such  lisT 

- cowwand  posts  or  air lift  control  cen t e rs  ( ALCCs ) 

7.  Coordinate  revisions  for  special  operations  withTbase  flying 

. authorities . 

8.  Coordinate  revisions  for  special  ope ra t ions  w i  th host  nation 

- 9T  Federal  Aviation  Adwinistration  (FAA)  agencies 

9.  Design  nap  overlays  ~ .  ’  ” 

10.  Determine  requirements  for  space,  personnel,  equipment,  or 

. . .  supplies 

11.  Determine  work  priorities  . . —  r-.— * 

12.  Develop  facility  reference  charts  . 

13 .  Develop  organ  izai Iona  1  charts  .  ~  H~77~  77  ““ 

14 .  Develop  work  methods  or  procedures 

15.  Draft  budget  or  financial  requirements  ”777  ~ 

16.  Establish  dispersed  controller  programs  ~ 

17.  Establish  facility  documentation  f  i les~~”  ;• :  ■: ; TT7~;  '  . ;  "7; . 7 • 7'7 

18.  Establish  facility  miss ionimpac t  guidelines  ““ 

19.  Establish  local  procedures  for  controlling  f  light  I:he ck  ~~~  ~ 

_ _ aircraft  .  :  ;  :• : : 

20.  Establish  organizational  policies,  operating  instructions 

- LPJsXi  ting,  procedu r es  ( SOPs ) 

21.  Establish  performance  standards  for  subordinates  7 

22.  Establish  priorities  for  restoring  equipment  ~~ 

23.  Establish  procedures^f^r wiit r^fing^ traffic  betwee^ectorsr 
or  other  facilities 
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2)  TPD  Instructions  and  Scale  Heading 
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INSTRUCTIONS 

FOR  RATING  TASK  PERFORMANCE  DIFFICULTY 


As  a  senior  technician,  you  have  been  selected  to  provide  needed  information 
pertaining  to  the  difficulty  of  tasks  performed  in  your  career  ladder.  This 
information  will  be  of  value  to  the  Air  Force  in  the  improvement  of  training, 
classification,  and  testing  programs.  To  accomplish  this  rating,  follow  the 
procedure  listed  below. 


NOTE:  To  obtain  the  maximum  response  possible,  it  is  requested  that  you  rate  each 
task  of  which  you  have  any  knowledge.  Rate  those  tasks  you  presently  perform  or 
supervise,  those  tasks  which  you  have  performed  at  a  prior  time  in  your  career,  and 
those  tasks  which  you  have  observed  or  supervised  while  being  performed  by 
others.  Most  personnel  with  your  experience  and  background  will  be  able  to  rate 
the  majority  of  the  tasks  listed  and  in  many  cases  to  rate  all  of  them. 

STEP  1.  Develop  a  frame  of  reference  for  rating  task  performance  difficulty.  To 
develop  a  frame  of  reference,  scan  the  entire  listing  of  tasks.  Pick  out  some  easy 
tasks  and  some  difficult  tasks.  Then,  find  some  tasks  which  fall  between  these 
extremes  that  are  of  average  difficulty.  Use  these  tasks  at  or  near  the  middle  of  the 
range  as  a  reference  point  for  judging  the  performance  difficulty  of  all  tasks  in  the 
inventory.  This  frame  of  reference  will  be  used  for  completing  STEP  2. 


STEP  2.  Estimate  the  task  performance  difficulty  rating  for  each  task  compared  with 
other  tasks  in  this  inventory.  Use  the  scale  shown  here  and  at  the  top  of  each  page 
to  rate  each  task. 


1.  Extremely  Low 

2.  Very  Low 

3.  Low 

4.  Below  Average 

5.  Average 

6.  Above  Average 

7.  High 

8.  Very  High 

9.  Extremely  High 


Begin  with  the  first  task  in  the  booklet  and  give  each  task  of  which  you  have 
knowledge  a  performance  difficulty  rating  from  1  to  9;  record  the  value  opposite  the 
task  statement  in  the  column  titled  'TASK  PERFORMANCE  DIFFICULTY."  Try  to  rate 
every  task  on  each  page.  Remember  (from  STEP  1)  that  you  are  comparing  each 
task  with  the  other  tasks  in  the  career  field. 

STEP  3.  The  last  page  of  the  booklet  is  available  to  add  any  tasks  you  do  now 
which  are  not  listed.  Your  constructive  suggestions  in  improving  the  job  inventory 
will  be  useful. 


STEP  4.  Review  the  booklet  to  see  that  you  have  rated  the  PERFORMANCE 
DIFFICULTY  of  all  tasks  possible.  Each  task  can  be  given  only  one  rating. 


APPENDIX  E:  EXAMPLE  OF  DUPLICATE  PAGE  SETUP 


1) 

2) 

3) 

4) 

5) 

6) 


AFSC  272X0  TPD  Page  10 
AFSC  272X0  TPD  Page  11 
AFSC  272X0  TPD  Page  12 
AFSC  272X0  TPD  Page  13 
AFSC  272X0  TPD  Page  11 
AFSC  272X0  TPD  Page  14 
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TASK  PERFORMANCE  DIFFICULTY  RATING 


INSTRUCTIONS 


AFSC 

272X 


Listed  below  are  a  duty  and  the  tasks  which  it  includes.  Rate  each  task  for 
performance  difficulty. 


TASK  PERFORMANCE 
DIFFICULTY 


1.  Extremely  Low 

2.  Very  Low 

3.  Low 

4.  Below  Average 

5.  Average 

6.  Above  Average 

7.  High 

8.  Very  High 

9.  Extremely  High 


F. 

PERFORMING  GENERAL  AIR  TRAFFIC  CONTROL  FUNCTIONS 

206. 

Activate  back-up  communications  systems 

207. 

Adjust  radar  scopes 

208. 

Annotate  and  update  flight  progress  strips 

209. 

Apply  military  assumes  responsibility  for  separation 
aircraft  (HARSA)  procedures 

of 

210. 

Apply  mode-c  veil  procedures 

211. 

Apply  nonradar  separation  procedures 

212. 

Apply  visual  separations 

. “213. 

Approve  aircraft  operations  in  airport  radar  surveillance 
areas  (ARSAs) 

2147' 

Approve  aircraft  operations  in  airport  traffic  areas 

(ATAs) 

2  is. 

Approve  aircraft  operations  in  control  zones  (CZs) 

216. 

Approve  aircraft  operations  in  terminal  control  areas 

(TCAs) 

”  2l7.  “ 

Approve  or  coordinate  aerial  refueling  operations 

— g— 

Approve  or  coordinate  aircraft  operations  in  warning 
restricted  areas 

or 

219.  " 

Approve  or  coordinate  enroute  altitude  reservations 

*“  220“ 

Approve  or  coordinate  IFR  aircraft  departures 

221.  Approve  or  coordinate  fc>ilot  requests  for  deviations  from  ATC 

clearances _ _ _ _ 

222.  Approve  or  coordinate  special  visual  flight  rules  (SVFR) 

j. _  operations  . :  •  :‘-'7  ’  '  ‘  7 .  •  :  :  ; ; :=  :• . ( 

223.  Approve  or  disapprove  flight  join-ups 

224.  Approve  parachute ,  drop  zone,  or  airdrop  operations 

225.  Approve  pilot  and  supervisor  of  flying  (SOF)  communications 

_ interruptions  _  _  _ _ 


226.  Approve  pilots  request  to  conduct  unusual  maneuvers 

227.  Assign  frequencies  to  aircraft 


228.  Assign  transponder  modes  or  codes 
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TASK  PERFORMANCE  DIFFICULTY  RATING  |AFSC  I 

— - — -  1  272X0  |  Page  11  of  24  pan.r 

I  INSTRUCTIONS 

p.rf  or-.anee^di  f  f  icul  ty  *  **  '*'*'*'  “  lnclud«s-  Rat«  task  for 

TASK  PERFORMANCE 
difficulty 

X.  Extremely  Low 

2.  Very  Low 

3.  Low 

4.  Below  Average 

5.  Average 

6.  Above  Average 

7.  High 

8.  Very  High 

9.  Extremely  High 

>  < 

l  2  } 

j'j; . 

I  229.  Authorize  or  control  circling  maneuvers 

26 

|  ^ou.  Calculate  lowest  usable  flight  levels  . . 

1  231.  Clean  work  areas  or  equipment  . .  . ' " .  “ " 

- - - - -  - 

27 

28 

I  Zjz.  Complete  preduty  equipment  checklists 

“'29 

30 

I  Conduct  alternate  facility  procedures 

1  Conduct  briefings  on  ATC  equipment 

I  235.  Conduct  briefings  on  ATC  operations  . . “  . . 

*31 

32 

I  236.  Conduct  comraunicat ion-out  procedures  . 

~33 

34 

35 

|  zj/.  Conduct  procedures  for  opening  or  closing  runways 

i  zjb.  Confirm  aircraft  altitudes  .  ~ 

I  239.  Control  aircraft  with  hot  gun  or  hung  ordnance 

36 

‘37 

38 

«  zhu.  control  no-radio  (NORDO)  aircraft 

I  241.  Control  practice  precautionary  approaches 

a  Z4z.  Control  receiver-only  aircraft  “ 

|  243.  Coordinate  aircraft  handoffs  - - 

- - 

39 

46 

41 

42 

43 

44 

45 

1  2A4.  Coordinate  allied  altitude  reservation  requests 

I  z*o.  Coordinate  altitude  reservation  conflicts 

i  Z4b.  Coordinate  hazardous  cargo  operations 

1  im.  Coordinate  or  control  aircraft  surge  launch  and  recovery 

| _ _ ( ASLAR)  operations 

■  coordinate  or  control  priority  missions  ~ 

i  coordinate  or  control  special  tactical  missions 

46 

1  Coordinate  or  relay  SOF  requests 

47 

I  251.  Coordinate  search  and  rescue  operations 

48 

1  252.  Coordinate  status  of  ATC  facilities  with  other  agencies  T~  ~ 

49 
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TASK  PERFORMANCE  DIFFICULTY  RATING 


INSTRUCTIONS 


AFSC 

272Xi 


Listed  below  are  a  duty  and  the  tasks  which  it  includes.  Rate  each  task  for 
performance  difficulty. 


Page  12  of  24  Pages 


TASK  PERFORMANCE 
DIFFICULTY 


1.  Extremely  Low 

2.  Very  Low 

3.  Low 

4.  Below  Average 

5.  Average 

6 .  Above  Average 

7.  High 

8.  Very  High 

9.  Extremely  High 


253.  Coordinate  use  of  airspace  with  other  agencies  or  facilities 

254.  Coordinate  video  nap  alignments 

255.  Copy  or  issue  airfield  advisories 

256.  Copy  or  issue  field  operating  conditions 


257.  Copy  or  issue  meteorological  aviation  report  (METAR)  weather 

reports  _ _ _  _ _ _ _  _ _ _ , _ _ 

258 .  Copy  or  relay  standard  weather  observations 

259.  Copy  or  transmit  enroute  clearances  using  FAA  or 
International  Civil  Aviation  Organization  (ICAO)  procedures 

260.  Copy  or  transmit  enroute  clearances  using  host-nation 
agreement  procedures 

261.  Determine  aircraft  positions  using  nonradar  procedures 

262.  Direct  or  vector  aircraft  to  external  store  jettison  areas 

263.  Direct  or  vector  aircraft  to  fuel  dumping  areas 


264. 

Direct  or  vector  emergency  aircraft 

to  alternate  airports 

265. 

Erect  or  tear  down  TRN-41  mobile  tactical  air  navigation 
systems  (TACANs) 

266. 

Extract  information  from  transmitted 

flight  plans 

267. 

—  .~ 

File  flight  progress  strips 

Formulate  IFR  clearances 

269. 

Hold  arriving  VFR  aircraft  at  visual 

fixes 

270.  Inform  agencies  of  observed  unusual  events  dr  incidents 

271.  Initiate  antihijacking  procedures 

272.  Initiate  emergency  assistance  procedures 

273.  Initiate  emergency  handling  procedures  for  special  operations 


274.  Initiate  requests  for  control  of  airspace  from  other 

_ :  facillt i  es_ _ . / _  : 

275.  Initiate  special  control  actions  for  communist  aircraft 

_ flights _  _ 

276.  issue  advance  approach  information  to  arriving  aircraft 


50 

51 

52 

53 

54 


55 

56 


57 

58 

59 

60 
"61 
62 

63 

64 

65 

66 


67 

68 
"69 
70 

"71 

72 
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TASK  PERFORMANCE  DIFFICULTY  RATING 

INSTRUCTIONS 

w ■«  — ....  ,„r 


I  Pag*  13  of  24  Pages 

TASK  PERFORMANCE 
DIFFICULTY 


277.  Issue  aircraft  in-flight  fuel  dumping  advisories 


1.  Extremaly  Low 

2.  Very  Low 

3.  Low 

|4.  Below  Average 

5.  Average 

6 .  Above  Average 

7.  High 

8.  Very  High 

9.  Extremely  High 


Issue  aircraft  speed  adjustments 
Issue  alternate  clearances 

Issue  alt i me  ter  sett i n g  s . 

Issue  altitude  assignments 


Issue  bird  advisories . . 

283.  issue  clearance  void  times 

284.  issue  go-around  instructions 

285.  issue  iFR  holding  instructions 

286 .  Issue  row-altitude  alerts . 

287.  Issue  missed  approach  instructions 

288.  issue  traffic  advisories 

289.  Issue  wake  turbulence  advisories 


290.  Issue  warning  area  advisories 

291.  Issue  weather  advisories 


issue  wheels  down  advisories 

293 .  Issue  wind  *  advisor ies 


294.  Hake  time  checks  . 

295.  Monitor  assigned  frequencies . 

296.  Notify  agencies  of  runways  in  use 

297.  Operate  generators 

298.  Operate  land  lines 


299.  Operate  or  check  remote  status  indicators 

300.  Operate  or  check  time  announcers 
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TASK  PERFORMANCE  DIFFICULTY  RATING 


INSTRUCTIONS 

Listed  below  are  a  duty  and  the  tasks  which  it  includes.  Rate  each  task  for 
performance  difficulty. 

229. 

Authorize  or  control  circling  maneuvers 

230. 

Calculate  lowest  usable  flight  levels 

231. 

Clean  work  areas  or  equipment 

232. 

Complete  preduty  equipment  checklists 

233. 

Conduct  alternate  facility  procedures 

234. 

Conduct  briefings  on  ATC  equipment 

235. 

Conduct  briefings  on  ATC  operations 

236. 

Conduct  communication-out  procedures 

237. 

Conduct  procedures  for  opening  or  closing  runways 

238. 

Confirm  aircraft  altitudes 

239. 

Control  aircraft  with  hot  gun  or  hung  ordnance 

240. 

Control  no- radio  (NORDO)  aircraft 

241.  ' 

Control  practice  precautionary  approaches 

242. 

Control  receiver-only  aircraft 

243. 

Coordinate  aircraft  handoffs 

. ”  2447 

Coordinate  allied  altitude  reservation  requests 

245. . 

Coordinate  altitude  reservation  conflicts 

TASK  PERFORMANCE 
DIFFICULTY 


1.  Extremely  Low 

2.  Very  Low 

3 .  Low 

4.  Below  Average 

5.  Average 

6 .  Above  Average 

7.  High 

8.  Very  High 

9.  Extremely  High 


246. 

Coordinate 

hazardous  cargo  operations 

247. 

Coordinate 

or  control  aircraft  surge  iaunch  and  recovery 

(ASLAR)  operations  1 

248. 

Coordinate 

or  control  priority  missions 

249. 

Coordinate 

or  control  special  tactical  missions 

2567; 

Coordinate 

or  relay  SOF  requests 

251. 

Coordinate 

search  and  rescue  operations 

252. 

Coordinate 

status  of  ATC  facilities  with  other  agencies 
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TASK  PERFORMANCE  DIFFICULTY  RATING 


INSTRUCTIONS 

Listed  below  are  a  doty  and  the  tasks  which  it  includes.  Rate  each  task  for 
performance  difficulty. 


Operate  or  check  voice  recorders 

Operationally  check  automatic  terminal ''informaVioiri''sVfvTces' 
(AXIS) 

Participate  in  preduty  familiarization  briefings 

Participate  in  3i«uiat^''cra8h,‘'aiertV~oVlisiir8^Fcoi»troi™ 
exercises 

Perform  interfacility  coordinations 


Perform  intrafacility  coordinations  “~r  .  T™* - 

.  Perform  meaconing,  interference,’ ’jamming,'"  and  "intrusion 

_ (  MIJI)  p  roc ed u  res 

►  Perform  radar  beacon  checks 

Plot  altitude  reservations 

Practice  facility  evacuation  procedures  • 

Practice  security  control  of  air  traffic  and  air  navigation 
. aid^  (SCATANA)  procedures 

Prepare  altitude  reservation  requests  ' 

Prepare  ATIS  messages  for  transmissions 


Prepare  video  maps  . . .  ~ .  " 

Process  airspace  reservation  amendment  delays  or 
_ cance  l  la tl.ons_ 

Process  ATCALS  decommissioning  messages  :  "  ■  '."~T  . 

Provide  or  coordinate  special  controls  for" air  defense  * 
Intercept  J* is s  ions 

Provide  special  handling'  for  a ircraf t  NAVAib  flight--”'  :  ~ 

inspections 

Provide  special  handling  for  special  operations  aircraft 

Provide  special  handling  for  very  importantHbersons  (VlPsVor 
,  coded  aircraft  ■  f;  ;  . 

Provide  wake  turbulence  separations  *'*  "* 

Rece  ive  or  re  lay  •nrout^"iir"  traf f ic~iovement  .in£onMit5«rTr~ 

Regulate  flow  of  traffic  between  s ec tors  or  f ac il it ies“*  - 

Relay  aircraft  arrival  or  departure  ti»os  ^  77T'~T:_-T~'''‘'-r'" 


TASK  PERFORMANCE 
DIFFICULTY 

1.  Extremely  Low 

2.  Very  Low 

3.  Low 

4.  Below  Average 

5.  Average 

6.  Above  Average 

7.  High 

8.  Very  High 

9.  Extrenely  High 
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INSTRUCTIONS  FOR  COMPLETING  THE  DUTY-TASK  SECTION 

READ  THIS  PAGE  BEFORE  GOING  FURTHER 


Have  you  completed  the  Background  Information  Section? 
Make  sure,  before  you  continue  with  this  procedure. 


I 


PROCEDURE  A.  CHECKING  TASKS  OF  PRESENT  JOB 

1.  As  you  read  each  task  in  the  Duty-Task  section,  place  a  check  beside  each  task  you  perform  in  your  present 
job.  Do  not  mark  tasks  you  have  performed  in  previous  jobs,  or  tasks  you  feel  you  are  qualified  to  perform.  Do  not 
confuse  work  you  do  yourself  with  work  you  supervise.  Mark  only  those  tasks  you  actually  perform  in  your  present 
job.  Put  your  check  mark  in  the  column  headed  “Check-lf  Done  Now".  When  you  have  checked  ALL  tasks 
performed  in  your  present  job,  return  to  this  page  and  follow  "Procedure  B"  below. 

2.  DO  NOT  COMPLETE  THE  RIGHT-HAND  COLUMN  AT  THIS  TIME. 

3.  If  a  task  you  perform  is  not  listed  anywhere  in  the  entire  list,  add  it  on  the  “Write-In  Comments"  page  at  the  back 
of  this  booklet.  Do  not  add  tasks  that  are  classified. 

4.  Remember,  at  this  time  you  are  to  complete  only  the  column  headed  “Check-lf  Done  Now".  Please  go  to  the 
next  page  and  begin  checking  those  tasks  you  perform  in  your  present  job. 

PROCEDURE  B.  RATING  TIME  SPENT  ON  TASKS  IN  PRESENT  JOB 

1.  Have  you  checked  each  task  you  perform  in  your  present  job?  Make  sure,  before  you  continue  with  this 
procedure. 

2.  Now  you  are  to  rate  the  relative  amount  of  time  you  spend  performing  each  task  in  your  present  job.  Relative 
time  spent  means  the  total  time  you  spend  doing  the  task  compared  with  the  time  you  spend  on  each  of  the  other 
tasks  of  your  present  job. 

3.  Use  a  rating  of  “1"  if  you  spend  a  “very  small  amount”  of  time  on  a  task.  Use  a  rating  of  “2"  for  "much  below 
average”  time,  and  so  on,  up  to  a  rating  of  "9”  if  you  spend  a  “very  large  amount"  of  time  on  the  task. 

4.  Remember,  you  are  to  rate  only  those  tasks  that  you  have  already  checked  in  the  "Check-lf  Done  Now”  column. 


i 


5.  Place  your  rating,  according  to  the  9-point  scale,  in  the  right-hand  column  headed  "Time  Spent  Present  Job”  by 
blackening  the  appropriate  circle.  When  marking  your  responses,  care  should  be  taken  not  to  overlap  into  other 
ovals  on  the  same  line. 
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6.  When  you  have  completed  all  your  ratings  in  the  right-hand  column,  you  will  have  completed  this  USAF  Job 
Inventory  and  you  may  turn  it  in  to  your  Occupational  Survey  Control  Officer. 

7.  Now,  go  to  the  next  page  and  begin  rating  the  "Time  Spent”  on  those  tasks  you  checked  previously. 
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DUTY-TASK  SECTION 


1.  CHECK  tasks  you  perform  now. 

2.  If  you  DON’T  do  it,  DON’T  check  it. 


3.  In  the  ’'time  spent”  column  RATE  all  checked  tasks  on  time  spent 
in  present  job.  If  you  CHECKED  it,  RATE  it. 


A.  ORGANIZING  AND  PLANNING 


1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 

11. 

12. 

13. 

14. 

15. 

16. 

17. 

18. 

19. 

20. 

21. 

22. 

23. 

24. 

25. 

26. 

27. 

28. 

29. 

30. 

31. 

32. 

33. 

34. 

35. 

36. 

37. 

38. 

39. 

40. 

41. 

42. 


Assign  additional  duties . 

Assign  personnel  to  duty  positions  . . . 

Assign  sponsors  for  newly  assigned  personnel  . . 

Coordinate  air  traffic  control  (ATC)  procedures  for  disaster  control  exercises 

with  other  agencies  . . * . . 

Coordinate  aircraft  control  priorities  with  base  flying  authorities  . 

Coordinate  airlift  operations  with  other  agencies,  such  as  command  posts  or 

airlift  control  centers  (ALCCs)  . .  . . 

Coordinate  revisions  for  special  operations  with  base  flying  authorities  .  . . 

Coordinate  revisions  for  special  operations  with  host  nation  or  Federal  Aviation 

Administration  (FAA)  agencies  . 

Design  map  overlays  . * . 

Determine  requirements  for  space,  personnel,  equipment,  or  supplies  . 

Determine  work  priorities  . ♦ . 

Develop  facility  reference  charts  . 

Develop  organizational  charts  . 

Develop  work  methods  or  procedures  . 

Draft  budget  or  financial  requirements . 

Establish  dispersed  controller  programs  . . . 

Establish  facility  documentation  files  . 

Establish  facility  mission  impact  guidelines  . 

Establish  local  procedures  for  controlling  flight  check  aircraft  . . 

Establish  organizational  policies,  operating  instructions  (Ols),  or  standing 

operating  procedures  (SOPs)  . 

Establish  performance  standards  for  subordinates . 

Establish  priorities  for  restoring  equipment  . 

Establish  procedures  for  controlling  traffic  between  sectors  or  other  facilities  . 

Establish  procedures  for  notifying  other  agencies  of  equipment  failures  . 

Establish  publication  libraries  . . 

Plan  briefings  . 

Plan  duty  schedules  . . 

Plan  facility  equipment  layouts . 

Plan  for  ATC  analysis  visits  . 

Plan  layouts  of  facilities  . 

Plan  safety  programs  . 

Plan  security  programs  . 

Plan  staff  assistance  or  site  visits  . 

Plan  work  assignments  . . 

Prepare  facility  checklists  . . 

Prepare  job  descriptions  . 

Prepare  minimum  vectoring  altitude  (MVA)  charts  . 

Prepare  or  submit  recommendations  for  improving  or  standardizing  ATC  procedures 
Prepare  recommendations  for  changes  to  ATC  and  landing  systems  (ATCALSs)  . .  . . 

Prepare  requests  for  unit  detail  listing  (UDL)  changes  . 

Prepare  unit  emergency  plans  . 

Prepare  work  requests  to  base  service  organizations  . 
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DUTY-TASK  SECTION 


1.  CHECK  tasks  you  perform  now. 

2.  If  you  DON’T  do  it,  DON’T  check  it. 


3.  In  the  “time  spent"  column  RATE  all  checked  tasks  on  time  spent 
in  present  job.  If  you  CHECKED  it,  RATE  it. 


43.  Schedule  ATC  meetings  . 

44.  Schedule  briefings  - . 

45.  Schedule  flight  physicals  . 

46.  Schedule  leaves  or  passes  . 

47.  Schedule  navigational  aid  (NAVAID)  flight  checks 


B.  DIRECTING  AND  IMPLEMENTING 


48. 

49. 

50. 

51. 

52. 

53. 

54. 

55. 

56. 

57. 

58. 

59. 

60. 
61. 
62. 

63. 

64. 

65. 

66. 

67. 

68. 

69. 

70. 

71. 

72. 

73. 

74. 

75. 

76. 

77. 

78. 

79. 

80. 
81. 
82. 

83. 

84. 


Approve  electrical  power  transfers  . 

Conduct  briefings  for  newly  assigned  pilots  on  ATC  policies  or  procedures 
Conduct  exercise  and  contingency  briefings  to  aircrews  on  local  ATC  policies  or 

procedures  . 

Conduct  facility  tours  . 

Conduct  staff  briefings  on  status  of  ATCALSs 

Conduct  staff  meetings  . 

Counsel  personnel  on  personal  or  military-related  problems 
Direct  conventional  air  route  traffic  control  activities  (ARTCCs) 

Direct  development  or  maintenance  of  status  boards,  graphs,  or  charts 
Direct  ground  controlled  approach  (GCA)  activities 
Direct  maintenance  of  administrative  files 
Direct  nonradar  approach  control  activities 
Direct  procedures  for  performing  equipment  checks 
Direct  radar  air  route  traffic  control  activities 
Direct  radar  approach  control  (RAPCON)  activities 
Direct  radar  final  approach  control  activities 

Direct  tower  instrument  flight  rules  (IFR)  approach  control  activities  . . 

Direct  tower  visual  flight  rules  (VFR)  control  activities . 

Implement  cost-reduction  programs . . .  . 

Implement  safety  programs  . 

Implement  security  programs  . 

Implement  suggestion  programs  . 

Initiate  corrective  actions  based  on  inspection  deficiency  reports  . 

Initiate  personnel  action  requests  . 

Interpret  ATC  policies  for  using  activities  . 

Interpret  policies,  directives,  or  procedures  for  subordinates . 

Maintain  ATC  liaison  with  foreign  nationals  . 

Maintain  ATC  liaison  with  other  United  States  agencies  . 

Maintain  contingency  plans  . 

Post  duty  schedules  . . 

Prepare  or  submit  recommendations  for  changes  to  ATC  publications,  other  than 
base  . 
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Prepare  or  submit  recommendations  for  changes  to  base  ATC  publications  . 

Prepare  or  submit  terminal  instrument  procedures  (TERPS)  packages  . 

Review  TERPS  packages  . 

Revise  facility  equipment  layouts . 

Submit  changes  to  flight  information  publications  (FLIP)  . . 

Supervise  Air  Traffic  Control  Automated  Systems  Programming  Specialists  (AFSC 
D272X0)  . 
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DUTY-TASK  SECTION 


rnmnim 


1.  CHECK  tasks  you  perform  now. 

2.  If  you  DON'T  do  it,  DON’T  check  it. 


\W 


3.  In  the  “time  spent"  column  RATE  all  checked  tasks  on  time  spent  \  v 
in  present  job.  If  you  CHECKED  it,  RATE  it.  Y* 


85.  Supervise  Air  Traffic  Control  Managers  (CEM  27200)  . 

86.  Supervise  Air  Traffic  Control  Operators  (AFSC  27250)  . 

87.  Supervise  Air  Traffic  Control  Superintendents  (AFSC  27290)  . 

88.  Supervise  Air  Traffic  Control  Technicians  (AFSC  27270) . * . 

89.  Supervise  Apprentice  Air  Traffic  Control  Operators  (AFSC  27230)  . 

90.  Supervise  Apprentice  Combat  Control  Operators  (AFSC  27330)  . 

91.  Supervise  civilian  personnel  . 

92.  Supervise  Combat  Control  Operators  (AFSC  27350) . 

93.  Supervise  Combat  Control  Technicians  (AFSC  27370)  . . . 

94.  Supervise  foreign  nationals  . 

95.  Supervise  military  personnel  with  AFSCs  other  than  272X0  . 

96.  Write  general  correspondence  . 


MMttiMlNNMNl 

pa 

mm 


^mmmehkmmI 

■MHHHMHM 


C.  INSPECTING  AND  EVALUATING 

Administer  facility  rating  examinations  or  position  certifications  . 

Conduct  ATC  analysis  visits  . 

Conduct  ATC  facility  self-inspections  . 

Conduct  on-the-job  performance  evaluations  of  air  traffic  controllers  . 

Conduct  staff  assistance  or  site  visits  . 

Evaluate  activity  reports  . 

Evaluate  aircraft  save  requests  . 

Evaluate  ATC  complaints  . 

Evaluate  ATC  methods  or  techniques . 

Evaluate  ATC  operations  reports  .  . . 

Evaluate  ATC  problem  areas  . 

Evaluate  ATC  recommendations  . 

Evaluate  ATC  voice-recorder  tapes  pertaining  to  aircraft  accidents  or  incidents 

Evaluate  ATC  withdraw  packages . . 

Evaluate  budget  or  financial  requirements  . 

Evaluate  individuals  for  promotion,  demotion,  or  reclassification  . 

Evaluate  inspection  reports  or  procedures  .  . . 

Evaluate  job  descriptions  . 

Evaluate  letters  of  agreement  (LOAs)  . . . 

Evaluate  maintenance  or  use  of  workspace,  equipment,  or  supplies  . 

Evaluate  memorandums  of  understanding  (MOUs) . 

Evaluate  mission  impact  resulting  from  ATCALS  deficiencies  . 

Evaluate  Ols  . 

Evaluate  operations  letters  . 

Evaluate  procedures  for  storage,  inventory,  or  inspection  of  property  items  . . 

Evaluate  safety  programs  . . . - . 

Evaluate  security  programs  . . 

Evaluate  suggestions  . 

Evaluate  unit  emergency  plans  . 

Evaluate  work  schedules  . . 

Evaluate  workload  requirements  . 

Implement  task  qualification  training  (TOT)  reports  . 
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DUTY-TASK  SECTION 


:  SPENT 


1.  CHECK  tasks  you  perform  now. 

2.  If  you  DON'T  do  it,  DON'T  check  It. 

3.  |n  the  “time  spent"  column  RATE  all  checked  tasks  on  time  spent 
In  present  job.  If  you  CHECKED  it,  RATE  it. 


129.  Inventory  equipment,  tools,  or  supplies,  other  than  mobile  equipment,  tools,  or 

supplies  . ; . 

130.  Investigate  accidents  or  incidents,  other  than  aircraft  accidents  or  incidents 

131.  Investigate  aircraft  accidents  or  incidents  . 

132.  Prepare  enlisted  performance  reports  (EPRs)  or  letters  of  evaluation  (LOEs) 

133.  Prepare  recommendations  for  awards  or  decorations  . 

134.  Write  civilian  performance  ratings  or  supervisory  appraisals 

135.  Write  staff  studies,  surveys,  or  special  reports . 


D.  TRAINING 


136. 

137. 

138. 

139. 

140. 

141. 

142. 

143. 

144. 

145. 

146. 

147. 

148. 

149. 

150. 

151. 

152. 

153. 

154. 

155. 

156. 

157. 

158. 

159. 

160. 
161. 
162. 

163. 

164. 

165. 

166. 

167. 

168. 

169. 

170. 

171. 

172. 


Administer  tests . . 

Assign  on-the-job  training  (OJT)  trainers  . . . 

Assign  resident  course  instructors  . 

Conduct  Air  National  Guard  or  Air  Force  Reserves  ATC  training  . 

Conduct  air  route  traffic  control  center  proficiency  training  . . 

Conduct  ATC  exercises  .  . 

Conduct  ATC  training  for  civilian  personnel  . 

Conduct  ATC  training  for  foreign  nationals . 

Conduct  ATCALS  equipment  training  . 

Conduct  briefings  on  new  training  techniques  . 

Conduct  control  tower  proficiency  training  . 

Conduct  facility  rating  training  . 

Conduct  ground  control  appraoch  (GCA)  proficiency  training  . 

Conduct  M-series  vehicle  training  . 

Conduct  OJT  . . . . . •. . 

Conduct  radar  final  control  proficiency  training  . 

Conduct  RAPCON  proficiency  training  . 

Conduct  resident  course  training  . 

Conduct  TERPS  specialist  training  . . . 

Counsel  trainees  on  training  progress  . 

Demonstrate  how  to  locate  technical  information . 

Determine  resident  course  training  requirements  .  . 

Develop  job  qualification  standards  (JOSs)  . 

Develop  JOS  training  references  . 

Develop  resident  course  curriculum  materials  . 

Direct  or  implement  OJT  programs  . 

Direct  or  implement  training  programs,  other  than  OJT  . . 

Establish  indoctrination  programs  for  newly  assigned  personnel  . 

Establish  study  reference  files  . 

Establish  unit  training  requirements  . 

Evaluate  individuals  for  specialized  training .  . 

Evaluate  OJT  trainers  . 

Evaluate  progress  of  resident  course  students  . 

Evaluate  training  methods  or  techniques  . 

Maintain  training  records,  charts,  or  graphs . 

Operate  maintenance  management  information  and  control  system  (MMICS)  terminals 
Participate  in  USAF  graduate  evaluation  programs  . 
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DUTY-TASK  SECTION 


MIM 


1.  CHECK  tasks  you  perform  now.  \  <5 

2.  If  you  DON’T  do  it,  DON’T  check  it.  > 

3.  In  the  “time  spent"  column  RATE  all  checked  tasks  on  time  spent 
in  present  job.  If  you  CHECKED  it,  RATE  it. 


vM 


173.  Perform  core  automated  maintenance  system  (CAMS)  duties  . 

174.  Plan  OJT  . 

175.  Prepare  course  control  documents . 

176.  Prepare  facility  rating  guides  . 

177.  Prepare  quarterly  training  reports  . . . 

178.  Prepare  recommendations  for  standardization  of  facility  procedures 

179.  Prepare  training  schedules . 

180.  Procure  training  aids,  space,  or  equipment  . 

181.  Schedule  personnel  for  training  . 

182.  Score  tests . 

183.  Update  facility  rating  suspense  files . 

1 84.  Write  test  questions  . 

185.  Write  training  reports . 


E.  PERFORMING  AIR  TRAFFIC  CONTROL  ADMINISTRATIVE  FUNCTIONS 


186.  Complete  TERPS  forms  . 

187.  Implement  dispersed  controller  taskings  .  .  . 

188.  Justify  manpower  authorizations  . 

189.  Maintain  facility  status  boards  . 

190.  Maintain  NAVAID  status  charts  . 

191.  Maintain  personnel  information  cards  (PICs) 

192.  Maintain  records  of  facility  operations  . 

193.  Prepare  ATC  evaluation  record  forms  . 

194.  Prepare  ATC  withdrawal  packages  . 

195.  Prepare  duty  schedules  . 

196.  Prepare  FAA  facility  certification  applications 

197.  Prepare  facility  reports  . 

198.  Prepare  LOAs  . 

199.  Prepare  MOUs  . 

200.  Prepare  Ols  . 

201.  Prepare  operations  tetters  . 

202.  Prepare  or  review  service  evaluation  reports 

203.  Prepare  security  inspection  forms  . 

204.  Prepare  standardization  check  forms  . 

205.  Review  manpower  authorizations  . 


F.  PERFORMING  GENERAL  AIR  TRAFFIC  CONTROL  FUNCTIONS 


206.  Activate  back-up  communications  systems  . 

207.  Adjust  radar  scopes  . . 

208.  Annotate  and  update  flight  progress  strips  . 

209.  Apply  military  assumes  responsibility  for  separation  of  aircraft  (MARSA) 

procedures  . 

210.  Apply  mode-c  veil  procedures  . 

211.  Apply  nonradar  separation  procedures  . 

212.  Apply  visual  separations  . 

213.  Approve  aircraft  operations  in  airport  radar  surveillance  areas  (ARSAs) 
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DUTY-TASK  SECTION 


1.  CHECK  tasks  you  perform  now. 

2.  If  you  DON’T  do  it,  DON’T  check  it. 


3.  In  the  “time  spent”  column  RATE  all  checked  tasks  on  time  spent 
in  present  job.  If  you  CHECKED  it.  RATE  it. 


214. 

215. 

216. 

217. 

218. 

219. 

220. 
221. 
222. 

223. 

224. 

225. 

226. 

227. 

228. 

229. 

230. 

231. 

232. 

233. 

234. 

235. 

236. 

237. 

238. 

239. 

240. 

241. 

242. 

243. 

244. 

245. 

246. 

247. 

248. 

249. 

250. 

251. 

252. 

253. 

254. 

255. 

256. 

257. 

258. 

259. 

260. 


Approve  aircraft  operations  in  airport  traffic  areas  (ATAs) . 

Approve  aircraft  operations  in  control  zones  (CZs)  . 

Approve  aircraft  operations  in  terminal  control  areas  (TCAs)  . 

Approve  or  coordinate  aerial  refueling  operations  . 

Approve  or  coordinate  aircraft  operations  in  warning  or  restricted  areas  .  .  . 

Approve  or  coordinate  enroute  altitude  reservations  . 

Approve  or  coordinate  IFR  aircraft  departures  .  . . 

Approve  or  coordinate  pilot  requests  for  deviations  from  ATt  clearances  . .  . 

Approve  or  coordinate  special  visual  flight  rules  (SVFR)  operations  . 

Approve  or  disapprove  flight  join-ups  . 

Approve  parachute,  drop  zone,  or  airdrop  operations  . 

Approve  pilot  and  supervisor  of  flying  (SOF)  communications  interruptions  .  . 

Approve  pilots'  request  to  conduct  unusual  maneuvers  . 

Assign  frequencies  to  aircraft  . . . 

Assign  transponder  modes  or  codes  . 

Authorize  or  control  circling  maneuvers  . 

Calculate  lowest  usable  flight  levels  . 

Clean  work  areas  or  equipment  . 

Complete  preduty  equipment  checklists  . 

Conduct  alternate  facility  procedures  . . 

Conduct  briefings  on  ATC  equipment  . 

Conduct  briefings  on  ATC  operations  . 

Conduct  communication-out  procedures . 

Conduct  procedures  for  opening  or  closing  runways  . 

Confirm  aircraft  altitudes  . 

Control  aircraft  with  hot  gun  or  hung  ordnance  . 

Control  no-radio  (NOR DO)  aircraft  . . . 

Control  practice  precautionary  approaches  . 

Control  receiver-only  aircraft  . . . 

Coordinate  aircraft  handoffs . 

Coordinate  allied  altitude  reservation  requests  . 

Coordinate  altitude  reservation  conflicts  . 

Coordinate  hazardous  cargo  operations . 

Coordinate  or  control  aircraft  surge  launch  and  recovery  (ASLAR)  operations 

Coordinate  or  control  priority  missions  . 

Coordinate  or  control  special  tactical  missions  . 

Coordinate  or  relay  SOF  requests  . 

Coordinate  search  and  rescue  operations  . 

Coordinate  status  of  ATC  facilities  with  other  agencies  . . 

Coordinate  use  of  airspace  with  other  agencies  or  facilities . 

Coordinate  video  map  alignments  . 

Copy  or  issue  airfield  advisories  . 

Copy  or  issue  field  operating  conditions . 

Copy  or  issue  meteorological  aviation  report  (METAR)  weather  reports  . 

Copy  or  relay  standard  weather  observations  . 

Copy  or  transmit  enroute  clearances  using  FAA  or  International  Civil  Aviation 

Organization  (ICAO)  procedures  . 

Copy  or  transmit  enroute  clearances  using  host-nation  agreement  procedures 
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DUTY-TASK  SECTION 


1.  CHECK  tasks  you  perform  now. 

2.  If  you  DON'T  do  it,  DON’T  check  it. 
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3.  In  the  “time  spent"  column  RATE  all  checked  tasks  on  time  spent 
in  present  job.  If  you  CHECKED  it,  RATE  it 
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Determine  aircraft  positions  using  nonradar  procedures  . 

Direct  or  vector  aircraft  to  external  store  jettison  areas  . 

Direct  or  vector  aircraft  to  fuel  dumping  areas  . . 

Direct  or  vector  emergency  aircraft  to  alternate  airports  . 

Erect  or  tear  down  TRN-41  mobile  tactical  air  navigation  systems  (TACANs) 

Extract  information  from  transmitted  flight  plans  . 

File  flight  progress  strips  . 

Formulate  IFR  clearances  . 

Hold  arriving  VFR  aircraft  at  visual  fixes  . 

Inform  agencies  of  observed  unusual  events  or  incidents  . 

Initiate  antihijacking  procedures  . 

Initiate  emergency  assistance  procedures  . 

Initiate  emergency  handling  procedures  for  special  operations  . 

Initiate  requests  for  control  of  airspace  from  other  facilities  . 

Initiate  special  control  actions  for  communist  aircraft  flights  . 

Issue  advance  approach  information  to  arriving  aircraft  . . 

Issue  aircraft  in-flight  fuel  dumping  advisories  . . . 

Issue  aircraft  speed  adjustments  . 

Issue  alternate  clearances  . 

Issue  altimeter  settings.  . 

Issue  altitude  assignments  . 

Issue  bird  advisories  . 

Issue  clearance  void  times . 

Issue  go-around  instructions  . . 

Issue  IFR  holding  instructions  . 

Issue  low-altitude  alerts  . 

Issue  missed  approach  instructions  . 

Issue  traffic  advisories  . . 

Issue  wake  turbulence  advisories  . 

Issue  warning  area  advisories  . . . 

Issue  weather  advisories  . .  . 

Issue  wheels  down  advisories  . *  • 

Issue  wind  advisories . 

Make  time  checks  . 

Monitor  assigned. frequencies  . 

Notify  agencies  of  runways  in  use  . 

Operate  generators  . 

Operate  landlines . 

Operate  or  check  remote  status  indicators . 

Operate  or  check  time  announcers  . 

Operate  or  check  voice  recorders  . 

Operationally  check  automatic  terminal  information  services  (ATIS)  . 

Participate  in  preduty  familiarization  briefings  . 

Participate  in  simulated  crash,  alert,  or  disaster  control  exercises . 

Perform  interfacility  coordinations  . 

Perform  intrafacility  coordinations  . 

Perform  meaconing,  interference,  jamming,  and  intrusion  (MIJI)  procedures 
Perform  radar  beacon  checks . . 
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DUTY-TASK  SECTION 


1.  CHECK  tasks  you  perform  now. 

2.  If  you  DON’T  do  it,  DONT  check  it. 

3.  In  the  "time  spent”  column  RATE  all  checked  tasks  on  time  spent 
in  present  job.  If  you  CHECKED  it,  RATE  it. 


352. 

353. 

354. 

355. 

356. 

357. 

358. 

359. 

360. 

361. 

362. 

363. 

364. 

365. 

366. 

367. 

368. 

369. 

370. 

371. 

372. 

373. 

374. 

375. 

376. 

377. 

378. 

379. 

380. 

381. 

382. 

383. 

384. 

385. 

386. 

387. 

388. 

389. 

390. 

391. 

392. 

393. 

394. 

395. 

396. 

397. 

398. 


Apply  radar  contact  tost  procedures  . 

Approve  or  issue  radar  pointouts . 

Authorize  visual  separations  between  IFR  arrivals  or  departures 

Conduct  military  training  route  (MTR)  operations . 

Confirm  aircraft  identifications  . . . 

Construct  voice  imprint  files  on  personal  computer  (PC)  radar  simulators  . . . 

Control  air  traffic  using  oceanic  procedures  . 

Control  airport  surveillance  radar  approaches  (ASRs)  . 

Control  instrument  approaches  . 

Control  precision  radar  approaches  (PARs)  . 

Control  timed  approaches  . . . 

Coordinate  aircraft  movement  and  identification  information  with  air  defense 

facilities . . 

Coordinate  aircraft  position  with  tower  or  runway  supervisory  units  (RSUs)  . 

Coordinate  and  relay  observed  radar  weather  with  other  agencies  . 

Coordinate  approach  or  landing  sequences  . . - . 

Forward  airport  lighting  requests  to  control  towers  . . 

Identify  aircraft  using  beacon  methods  . 

Identify  aircraft  using  primary  radar  methods  . 

Implement  air  defense  identification  zone  (ADIZ)  procedures  . . . 

Inform  aircraft  of  radar  identification  status  . 

Initiate  PC  radar  simulator  sessions  . 

Initiate  special  radar  actions  during  electronic  combat  activities  . 

Initiate  T-4  radar  simulator  sessions  . . 

Interpret  radar  beacon  displays  . . . 

Issue  approach  clearances . 

Issue  dimbout  instructions .  . 

Issue  decision  height  (DH)  advisories  . 

Issue  entry  or  exit  clearances  to  special  use  airspace  . 

Issue  minimum  descent  altitude  (MOA)  advisories  . 

Issue  radar  surveillance  approach  recommended  altitudes  . 

Monitor  special  use  airspace  . 

Operate  pseudopilot  consoles  . 

Operate  special  circuits  . 

Operationally  check  or  adjust  decision  height  lines  on  radar  indicators  . 

Operationally  check  or  adjust  equipment  alignment  voltages  . . 

Operationally  check  or  adjust  remote  line  amplifiers  (RLAs)  . 

Operationally  check  primary  radars  . 

Operationally  check  radar  antenna  tilt  meters  . 

Operationally  check  radar  reflector  displays  . 

Operationally  check  secondary  radars  . 

Perform  map  overlay  adjustments  . 

Perform  radar  equipment  turnaround  procedures  . 

Perform  radar  handoffs  . 

Program  scenarios  for  PC  radar  simulators  . 

Program  T-4  radar  simulator  tapes  . 

Provide  radar  advisories  to  VFR  aircraft  . 

Provide  radar  assistance  to  emergency  aircraft  . 
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DUTY-TASK  SECTION 


TIME  SPENT 


1.  CHECK  tasks  you  perform  now. 

2.  If  you  DON’T  do  it,  DON'T  check  it. 


3.  In  the  “time  spent”  column  RATE  all  checked  tasks  on  time  spent 
in  present  job.  If  you  CHECKED  it,  RATE  it. 


399.  Provide  radar  monitoring  of  instrument  approaches . 

400.  Provide  radar  separation  procedures  . 

401.  Provide  radar  service  for  arrivals  . 

402.  Provide  radar  service  for  departures  . , . 

403.  Provide  stage  radar  services  .  . . 

404.  Recognize  or  report  radar  malfunctions  . 

405.  Relay  enroute  aircraft  position  estimates . 

406.  Select  radar  beacon  presentations . . . 

407.  Track  or  flight-follow  unmanned,  suspect,  or  derelict  airborne  objects  . 

H.  PERFORMING  CONTROL  TOWER  FUNCTIONS 


409. 

410. 

411. 

412. 

413. 

414. 

415. 

416. 

417. 

418. 

419. 

420. 

421. 

422. 

423. 

424. 

425. 

426. 

427. 

428. 

429. 

430. 

431. 

432. 

433. 

434. 

435. 

436. 

437. 

438. 

439. 

440. 

441. 


Activate  emergency  evacuation  alarm  systems  . 

Activate  primary  crash  alarm  systems  . . 

Adjust  brite  radar  systems . 

Advise  pilots  of  observed  abnormal  aircraft  conditions  . 

Apply  reduced  runway  separation  criteria  . 

Apply  terminal  separation  procedures  . 

Approve  clearance  for  aircraft  or  vehicle  operations  in  NAVAID  critical  areas  . 

Assign  runways  for  landings  or  takeoffs . . . 

Authorize  intersection  takeoffs . 

Authorize  operations  in  visual  blind  spots  . 

Authorize  simultaneous  opposite  direction  operations  . 

Authorize  simultaneous  same  direction  operations  . 

Authorize  VFR  departures  . 

Clear  movement  areas  for  emergency  vehicle  operations  . 

Control  aircraft  using  light  gun  signals . 

Control  helicopter  operations . 

Control  simulated  flameout  (SFO)  approaches  . 

Control  taxiing  aircraft  . 

Control  vehicles,  equipment,  or  personnel  on  movement  areas  using  radios  or  light 

gun  signals  . . 

Coordinate  aircraft  maintenance  engine  runups . 

Coordinate  or  conduct  hydrazine  procedures  .  . 

Coordinate  runway  changes  . 

Establish  landing  sequences  . 

Issue  takeoff  or  landing  clearances  . 

Maintain  surveillance  of  airport  movement  or  traffic  areas  . 

Make  or  relay  limited  weather  observations  . 

Operate  aircraft  arresting  systems . . 

Operate  vehicle  traffic  control  signals  . 

Perform  departure  control  overrides  . 

Plot  or  relay  positions  on  crash  grid  maps  . 

Request  aircraft  releases  from  departure  control  . 

Select  appropriate  wind  sensors  . 

Support  minimum  interval  takeoff  operations  (MITOs)  . 

Verify  proper  arresting  system  configurations  . 
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DUTY-TASK  SECTION 


1.  CHECK  tasks  you  perform  now. 

2.  If  you  DON’T  do  it,  DON’T  check  it. 

3.  In  the  “time  spent"  column  RATE  all  checked  tasks  on  time  spent 
in  present  job.  If  you  CHECKED  it,  RATE  it 


I.  PERFORMING  MOBILE  OPERATIONS 


442. 

443. 

444. 

445. 

446. 

447. 

448. 

449. 

450. 

451. 

452. 

453. 

454. 

455. 

456. 

457. 

458. 

459. 

460. 

461. 

462. 

463. 

464. 

465. 

466. 

467. 

468. 

469. 

470. 

471. 

472. 

473. 

474. 

475. 

476. 

477. 


478. 

479. 

480. 

481. 

482. 

483. 


Compute  PAR  voltages  . 

Conduct  ATC  mobility  training  exercises  . 

Conduct  deployment  briefings  . 

Conduct  sight  survey  training  . 

Conduct  site  surveys  for  locating  mobile  ATCALS  . 

Construct  bunkers  or  revetment  walls  . 

Erect  or  tear  down  ATC  facilities  . 

Erect  or  tear  down  cantonment  facilities  . 

Erect  or  tear  down  communications  facilities  . 

Initiate  requests  for  notice  of  intent  to  construct  mobile  facilities 

Install  communications  landlines  . 

Inventory  mobile  equipment,  tools,  or  supplies  . 

Level  mobile  radar  equipment  . 

Level  mobile  tower  equipment  . 

Load  or  offload  equipment  from  aircraft  or  vehicles . 

Manage  dispersed  controller  programs  . 

Obtain  food,  lodging,  or  medical  support  for  mobile  teams  .... 

Obtain  weather  information  in  support  of  mobile  operations  . .  . 

Operate  M-series  vehicles  or  associated  equipment  . 

Operate  site  survey  equipment  . 

Palletize  mobile  equipment  for  airlift  . 

Perform  convoy  duties  . . . 

Perform  courier  duties 
Perform  operator  maintenance  on  M-series  vehicles  or  associated  equipment 
Perform  user  maintenance  of  site  survey  equipment 
Plan  aircraft  loading  or  unloading 
Position  mobile  ATC  equipment  or  support  equipment 
Prepare  mission  limiting  factors 
Prepare  mobile  functional  support  kits 
Prepare  TERPS  packages  for  mobile  site  development 
Process  airfield  waiver  packages 
Review  operation  orders  or  plans 

Set  up  disaster  control  facilities  . 

Set  up  mobile  team  ground  defense  facilities . 

Unpack  or  repack  communications  equipment  components 
Weigh  equipment  . . 


J.  PERFORMING  AIR  TRAFFIC  CONTROL  DATA  PROGRAMMER  FUNCTIONS 


Analyze  recorded  computer  data  for  aircraft  accident  or  incident  investigations 

Assemble  off-line  programs  . 

Assist  ATC  facility  managers  in  developing  operating  procedures  for  automated  ATC 

systems  software  sites  . 

Assist  training  sections  with  implementation  of  automated  scenario  problems  . 

Build  operational  programs  . 

Conduct  FAA  acceptance  testing  . . 
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DUTY-TASK  SECTION 


1.  CHECK  tasks  you  perform  now. 

2.  If  you  DON'T  do  it,  DON’T  check  it. 


3.  In  the  "time  spent”  column  RATE  all  checked  tasks  on  time  spent 
in  present  job.  If  you  CHECKED  it,  RATE  it. 


484. 

485. 

486. 

487. 

488. 

489. 

490. 

491. 

492. 

493. 

494. 

495. 

496. 

497. 

498. 

499. 

500. 

501. 

502. 

503. 


Conduct  in-house  tests  for  automated  ATC  systems  software 
Conduct  or  analyze  feasibility  studies  to  determine  hardware  or  software  needs  to 

support  automated  ATC  systems . . 

Conduct  tests  for  enroute  automated  radar  tracking  system  (EARTS)  software 
Configure  automated  ATC  systems  hardware  for  operational  needs 
Configure  EARTS  hardware  systems  for  operational  needs 
Document  and  record  data  for  computer  programs 
Interpret  system  performance  for  air  traffic  controllers 
Maintain  data  bases  for  automated  ATC  systems 

Maintain  data  bases  for  EARTS  . 

Perform  systems  analysis  to  resolve  software  problems 
Program  and  assign  peripheral  devices  in  data  processing  subsystems 

Program  digitized  ATC  geographic  maps  . 

Program  source  codes  for  software  maintenance  or  operational  programs  of 

automated  ATC  systems  . 

Program  ULTRA-30  language  code  for  software  maintenance  or  operational  programs 

of  EARTS  mainframe  . 

Provide  joint  service  or  interdepartmental  support  for  automated  ATC  systems 

implementations  . 

Submit  recommendations  to  FAA  for  software  changes  . 

Sustain  EARTS  software  to  meet  FAA  National  Air  Space  Configuration  Management 

software  requirements  . 

Test  software  for  related  computer  interfaces  . 

Validate  automated  ATC  systems  software  modifications 
Validate  EARTS  software  modifications  . 
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RETURN  TO  THE  INSTRUCTIONS  FOR  COMPLETING  THE  DUTY-TASK  SECTION  AND 
FOLLOW  'PROCEDURE  8* 


APPENDIX  G:  HIGH  PERCENT  TIME  SPENT  (PTS)  TASKS  WITH  TASK  FACTOR 

RANK  ORDERS 


1)  Paygrade  Personnel 

2)  Skill  Level  Personnel 

3)  TAFMS  Personnel 

4)  TICF  Personnel 
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Rank  Order  of  Task  Factor  Ratings  for  25  Tasks  Personnel  Spend  the  Greatest  Amount 

of  Time  Performing  by  Paygrade 


E-7/8/9 

TASKS 

RANK  ORDER 

TD  TLD  TPD 

E-4/5/6 

TASKS 

RANK  ORDER 

TD  TLD  TP 

D 

E-1/2/3 

TASKS 

RANK  ORDER 

TD  TLD  TPD 

B  96 

197 

279 

237 

F298 

490 

486 

494 

F298 

490 

486 

494 

B  54 

116 

213 

160 

F295 

500 

501 

502 

F295 

500 

501 

502 

All 

234 

295 

272 

F305 

405 

374 

419 

F231 

498 

502 

498 

Cl  32 

53 

65 

46  . 

F306 

500 

501 

502 

F305 

405 

374 

419 

Cl  07 

28 

92 

72 

F208 

398 

437 

454 

F306 

393 

388 

415 

B  88 

38 

203 

142 

F280 

499 

496 

496 

F208 

398 

437 

454 

B  73 

176 

207 

169 

F288 

499 

496 

496 

F280 

499 

496 

496 

Cl  33 

32 

64 

51 

F227 

491 

465 

489 

F227 

491 

465 

489 

B  72 

200 

202 

111 

F303 

491 

465 

489 

F288 

381 

358 

403 

A  20 

82 

95 

79 

F220 

484 

476 

483 

F228 

494 

466 

487 

B  86 

50 

118 

82 

F232 

484 

476 

483 

F220 

331 

280 

328 

C115 

93 

103 

108 

F267 

502 

495 

501 

F232 

484 

476 

483 

Cl  05 

39 

87 

123 

F228 

494 

466 

487 

F303 

489 

487 

482 

C120 

149 

195 

130 

F331 

350 

369 

418 

F331 

350 

369 

418 

C119 

96 

227 

122 

F293 

493 

488 

486 

F267 

502 

495 

501 

Cl  35 

17 

52 

33 

F231 

498 

502 

498 

F212 

312 

247 

317 

F306 

393 

388 

415 

F292 

498 

502 

498 

F293 

493 

488 

486 

C101 

37 

117 

117 

F212 

498 

502 

498 

F281 

466 

454 

465 

F298 

490 

486 

494 

F281 

466 

454 

465 

F292 

497 

490 

493 

F305 

405 

374 

419 

F345 

402 

299 

284 

F345 

420 

441 

466 

F295 

500 

501 

502 

F291 

428 

442 

436 

F243 

360 

372 

389 

A  21 

177 

215 

199 

F255 

461 

459 

457 

F324 

488 

491 

484 

Cl  08 

56 

144 

167 

F258 

472 

446 

450 

F296 

501 

498 

503 

A  1 

479 

493 

481 

F296 

501 

498 

503 

F258 

472 

446 

450 

A  14 

151 

93 

119 

F321 

185 

252 

280 

F289 

426 

433 

438 
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Rank  Order  of  Task  Factor  Ratings  for  25  Tasks  Experienced  Personnel  Spend  the 
Greatest  Amount  of  Time  Performing 


27270 

TASK 

RANK  ORDER 

TD  TLD  TPD 

27290 

TASK 

RANK  ORDER 

TD  TLD  TPD 

27200 

TASK 

RANK  ORDER 

TD  TLD  TPD 

F298 

190 

486 

494 

B  96 

197 

279 

237 

B  96 

197 

279 

237 

F305 

405 

374 

419 

Cl  35 

17 

52 

33 

Cl  35 

17 

52 

33 

B  86 

50 

118 

82 

A  21 

177 

215 

199 

A  20 

82 

95 

79 

F306 

393 

388 

415 

All 

234 

295 

272 

B  71 

347 

353 

309 

F295 

500 

501 

502 

Cl  32 

53 

65 

46 

C101 

37 

117 

117 

F303 

489 

487 

482 

B  54 

116 

213 

160 

B  73 

176 

207 

169 

B  96 

197 

279 

237 

A 10 

137 

99 

107 

B  72 

200 

202 

111 

F288 

381 

358 

403 

Cl  33 

32 

64 

51 

Cl  07 

28 

92 

72 

F208 

398 

437 

454 

A  34 

365 

378 

321 

Cl  32 

53 

65 

46 

F280 

499 

496 

496 

Cl  19 

96 

227 

122 

E205 

276 

132 

220 

F227 

491 

465 

489 

C120 

149 

195 

130 

Cl  33 

32 

64 

51 

F220 

331 

280 

328 

A  20 

82 

95 

79 

A  26 

363 

361 

262 

F293 

493 

488 

486 

A 14 

151 

93 

119 

Cl  05 

39 

87 

123 

F292 

497 

490 

493 

Cl  15 

93 

103 

108 

Cl  08 

56 

144 

167 

F232 

484 

476 

483 

Cl  26 

311 

363 

230 

A  38 

48 

158 

138 

F212 

312 

247 

317 

B  88 

38 

203 

142 

B  54 

116 

213 

160 

F228 

494 

466 

487 

A  27 

259 

286 

221 

A44 

430 

469 

406 

Cl  32 

53 

65 

46 

B  86 

50 

118 

82 

A43 

408 

468 

409 

F345 

420 

441 

466 

Cl  07 

28 

92 

72 

E200 

65 

102 

109 

H432 

341 

384 

353 

B  72 

200 

202 

111 

1443 

192 

74 

124 

F291 

428 

442 

436 

B  73 

176 

207 

169 

F235 

375 

419 

416 

A  2 

429 

479 

758 

A  46 

453 

439 

413 

All 

234 

295 

272 

F267 

502 

495 

501 

Cl  05 

39 

87 

123 

B  88 

38 

203 

142 

F281 

466 

454 

465 

A  38 

48 

158 

138 

C115 

93 

103 

108 

F321 

185 

252 

280 

El  88 

84 

51 

76 

Cl  20 

149 

195 

130 

95 


Rank  Order  of  Task  Factor  Ratings  for  25  Tasks  Senior  Air  Force  Personnel  (TAFMS) 
Spend  the  Greatest  Amount  of  Time  Performing 


97+ 

TASKS 

RANK  ORDER 

TD  TLD  TPD 

241+ 

TASKS 

RANK  ORDER 

TD  TLD  TPD 

F298 

490 

486 

494 

B  96 

197 

279 

237 

F306 

393 

388 

415 

C135 

17 

52 

33 

F305 

405 

374 

419 

A  20 

82 

95 

79 

F295 

500 

501 

502 

C101 

37 

117 

117 

F288 

381 

358 

403 

B  73 

176 

207 

169 

F303 

489 

487 

482 

All 

234 

295 

272 

B  86 

50 

118 

82 

B  72 

200 

202 

111 

F280 

499 

496 

496 

B  71 

347 

353 

309 

F208 

398 

437 

454 

Cl  07 

28 

92 

72 

F227 

491 

465 

489 

B  54 

116 

213 

160 

F220 

331 

280 

328 

C132 

53 

65 

46 

F232 

484 

476 

483 

Cl  05 

39 

87 

123 

B  96 

197 

279 

237 

B  88 

38 

203 

142 

F293 

493 

488 

486 

C133 

32 

64 

51 

F228 

494 

466 

487 

B  81 

2 

14 

42 

F292 

497 

490 

493 

B  83 

107 

165 

268 

F212 

312 

247 

317 

A  14 

151 

93 

119 

F281 

466 

454 

465 

C119 

96 

227 

122 

F267 

502 

495 

501 

C115 

93 

103 

108 

F291 

428 

442 

436 

Cl  08 

56 

144 

167 

F345 

420 

441 

466 

C120 

149 

195 

130 

F331 

350 

369 

418 

A  21 

177 

215 

199 

B  54 

116 

213 

160 

A  38 

48 

158 

138 

F255 

461 

459 

457 

A  34 

365 

378 

321 

F321 

185 

252 

280 

A  26 

363 

361 

262 

96 


Rank  Order  of  Task  Factor  Ratings  for  25  Tasks  Senior  Career  Field  (TICF)  Personnel 
Spend  the  Greatest  Amount  of  Time  Performing 


APPENDIX  H:.  HIGH  PERCENT  MEMBERS  PERFORMING  (PMP)  TASKS  WITH  TASK 

FACTOR  RANK  ORDERS 

1)  Paygrade  Personnel 

2)  Skill  Level  Personnel 

3)  TAFMS  Personnel 

4)  TICF  Personnel 


98 


Rank  Order  of  Task  Factor  Ratings  for  25  Tasks  Performed  by  the  Greatest  Percentage 

of  Paygrade  Personnel 


99 


Rank  Order  of  Task  Factor  Ratings  for  25  Tasks  Performed  by  the  Greatest  Percentage 

of  Experienced  Skill  Level  Personnel 


27270 

RANK  ORDER 

27290 

RANK  ORDER 

27200 

RANK  ORDER 

PMP 

TASK 

TD 

TLD 

TPD 

PMP 

TASK 

TD 

TLD 

TPD 

PMP 

TASK 

TD 

TLD 

TPD 

86 

F305 

405 

374 

419 

86 

Cl  32 

53 

65 

46 

93 

B  96 

197 

279 

237 

85 

F306 

393 

388 

415 

84 

B  96 

197 

279 

237 

80 

E205 

276 

132 

220 

85 

F298 

190 

486 

494 

84 

A  21 

177 

215 

199 

80 

C133 

32 

64 

51 

85 

F288 

381 

358 

403 

84 

B  54 

116 

213 

160 

80 

B  54 

116 

213 

160 

84 

F280 

499 

496 

496 

84 

C133 

32 

64 

51 

80 

A44 

430 

469 

406 

83 

F303 

489 

487 

482 

84 

A  46 

453 

439 

413 

80 

A  1 

479 

793 

481 

83 

F227 

491 

465 

489 

81 

A  1 

381 

358 

403 

73 

A  20 

82 

95 

79 

81 

F295 

500 

501 

502 

78 

All 

234 

295 

272 

73 

Cl  07 

28 

92 

72 

81 

F291 

428 

442 

436 

78 

C119 

96 

227 

122 

.73 

Cl  32 

53 

65 

46 

80 

F212 

312 

247 

317 

78 

A  20 

82 

95 

79 

73 

A  43 

408 

468 

409 

80 

F232 

484 

476 

483 

76 

Cl  20 

149 

195 

130 

73 

All 

234 

295 

272 

79 

F293 

493 

488 

486 

76 

A  14 

151 

93 

119 

73 

A  34 

365 

378 

321 

79 

F208 

398 

437 

454 

76 

Cl  15 

93 

103 

108 

67 

B  73 

176 

207 

169 

79 

F292 

497 

490 

493 

76 

B  88 

38 

203 

142 

67 

B  72 

200 

202 

111 

79 

F282 

474 

475 

471 

76 

A  27 

259 

286 

221 

67 

A  26 

363 

361 

262 

78 

F220 

331 

280 

328 

76 

El  95 

282 

310 

274 

67 

Cl  05 

39 

87 

123 

78 

F321 

185 

252 

280 

76 

B  77 

486 

494 

491 

67 

E200 

65 

102 

109 

78 

B  51 

487 

499 

497 

73 

A  10 

137 

99 

107 

67 

F235 

375 

419 

416 

78 

F255 

461 

459 

457 

73 

A  34 

365 

378 

321 

67 

B88 

38 

203 

142 

78 

F343 

440 

420 

437 

73 

Cl  26 

311 

363 

230 

67 

B75 

143 

152 

126 

78 

F348 

482 

484 

492 

73 

Cl  04 

58 

86 

84 

67 

E188 

84 

51 

76 

77 

F284 

427 

387 

396 

73 

A  3 

495 

503 

499 

67 

Cl  04 

58 

86 

84 

77 

F228 

494 

466 

487 

73 

F306 

393 

388 

415 

67 

A  3 

495 

503 

499 

77 

F294 

503 

500 

500 

73 

F227 

491 

465 

489 

67 

A  10 

137 

99 

107 

75 

F289 

426 

433 

438 

73 

F288 

381 

358 

403 

67 

C131 

4 

8 

19 

Rank  Order  of  Task  Factor  Ratings  for  25  Tasks  Performed  by  the  Greatest  Percentage 
of  Experienced  Air  Force  Personnel  (TAFMS) 


97+ 

RANK  ORDER 

241+ 

RANK  ORDER 

PMP 

TASK 

TD 

TLD 

TPD 

PMP 

TASK 

TD 

TLD 

TPD 

86 

F305 

405 

374 

419 

82 

B  96 

197 

279 

237 

86 

F306 

393 

388 

415 

75 

B  54 

116 

213 

160 

85 

F298 

190 

486 

494 

73 

Cl  33 

32 

64 

51 

85 

F288 

381 

358 

403 

72 

Cl  32 

53 

65 

46 

84 

F280 

499 

496 

496 

68 

B  88 

38 

203 

142 

84 

F227 

491 

465 

489 

68 

F235 

375 

419 

416 

83 

F295 

500 

501 

502 

68 

F288 

381 

358 

403 

83 

F303 

489 

487 

482 

68 

F306 

393 

388 

415 

82 

F291 

428 

442 

436 

68 

F227 

491 

465 

489 

80 

F293 

493 

488 

486 

67 

F305 

405 

374 

419 

80 

F232 

484 

476 

483 

67 

F280 

499 

496 

496 

79 

F208 

398 

437 

454 

65 

F298 

190 

486 

494 

79 

F212 

312 

247 

317 

65 

F282 

474 

475 

471 

79 

F282 

474 

475 

471 

63 

All 

234 

295 

272 

79 

F220 

331 

280 

328 

63 

A21 

177 

215 

199 

79 

F348 

482 

484 

492 

63 

A  1 

479 

493 

481 

79 

F255 

461 

459 

457 

63 

F295 

500 

501 

502 

78 

F228 

494 

466 

487 

63 

F212 

312 

247 

317 

78 

F284 

427 

387 

396 

63 

F293 

493 

488 

486 

77 

F321 

185 

252 

280 

63 

F291 

428 

442 

436 

77 

F292 

497 

490 

493 

63 

F284 

427 

387 

396 

77 

F294 

503 

500 

500 

62 

B  73 

176 

207 

169 

77 

F343 

440 

420 

437 

62 

C119 

96 

227 

122 

77 

F331 

350 

369 

418 

62 

F321 

185 

252 

280 

76 

F289 

426 

433 

438 

62 

F232 

484 

476 

483 
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Rank  Order  of  Task  Factor  Ratings  for  25  Tasks  Performed  by  the  Greatest  Percentage 
of  Senior  Career  Field  (TICF)  Personnel 


PMP 

97  + 

TASKS 

RANK  ORDER 

TD  TLD  TPD 

84 

F306 

393 

388 

415 

84 

F305 

405 

374 

419 

84 

F288 

381 

358 

403 

83 

F298 

190 

486 

494 

83 

F227 

491 

465 

489 

82 

F280 

499 

496 

496 

81 

F295 

500 

501 

502 

81 

F303 

489 

487 

482 

80 

F291 

428 

442 

436 

78 

F232 

484 

476 

483 

78 

F293 

493 

488 

486 

78 

F212 

312 

247 

317 

78 

F208 

398 

437 

454 

78 

F220 

331 

280 

328 

77 

F255 

461 

459 

457 

77 

F282 

474 

475 

471 

77 

F348 

482 

484 

492 

77 

F228 

494 

466 

487 

76 

F321 

185 

252 

280 

76 

B51 

487 

799 

497 

76 

F284 

427 

387 

396 

76 

F292 

497 

490 

493 

75 

F343 

440 

420 

437 

75 

F294 

503 

500 

500 

74 

F331 

350 

369 

418 

102 
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APPENDIX  I:  RANK  ORDER  OF  FACTOR  RATINGS  OF  TASKS  RATED  HIGH  ON 
TRAINING  EMPHASIS  AND  PERFORMED  BY  GREATER  THAN  50  PERCENT  OF 

FIRST  JOB  PERSONNEL 


TE 

M+SD* 

1st  JOB 

PMP 

TASKS 

RANK  ORDER 

TD  TLD  TPD 

7.67 

93 

F288 

381 

358 

403 

7.54 

86 

F289 

426 

433 

438 

7.43 

55 

F286 

262 

281 

375 

7.42 

88 

F212 

312 

247 

317 

7.33 

86 

F321 

185 

252 

280 

7.31 

72 

F243 

360 

372 

389 

7.21 

89 

F208 

398 

437 

454 

7.17 

87 

F220 

331 

280 

328 

7.17 

87 

F292 

497 

490 

493 

7.13 

50 

F347 

317 

262 

367 

7.10 

81 

F284 

427 

387 

396 

7.06 

83 

F281 

466 

454 

465 

7.00 

79 

F240 

153 

243 

168 

7.00 

69 

F272 

136 

197 

171 

6.98 

88 

F331 

350 

369 

418 

6.96 

90 

F291 

428 

442 

436 

6.88 

91 

F293 

493 

488 

486 

6.85 

95 

F305 

405 

374 

419 

6.83 

93 

F306 

393 

388 

415 

6.83 

75 

F238 

467 

463 

452 

6.81 

68 

F222 

232 

169 

242 

6.79 

91 

F227 

491 

465 

489 

6.75 

72 

F242 

221 

254 

231 

6.71 

72 

F329 

462 

467 

472 

6.67 

90 

F282 

474 

475 

471 

*3.22+2.27=5.49 
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